Device, method, and graphical user interface for battery management

ABSTRACT

A device monitors a battery charge level and battery-usage patterns and provides alerts in accordance therewith.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/407,590 entitled “Device, method, and graphical user interface forestablishing a relationship and connection between two devices,” filedon May 9, 2019, which is a continuation of U.S. patent application Ser.No. 14/863,069 entitled “Device, method, and graphical user interfacefor establishing a relationship and connection between two devices,”filed on Sep. 23, 2015, which claims priority to U.S. Provisional PatentApplication No. 62/111,100, entitled “Device, method, and graphical userinterface for establishing a relationship and connection between twodevices,” filed on Feb. 2, 2015, which are hereby incorporated byreference in their entireties.

This is related to the following applications: U.S. Provisional PatentApplication No. 61/832,842, filed Jun. 8, 2013, entitled “Device,Method, and Graphical User Interface for Synchronizing Two or MoreDisplays”; U.S. Provisional Application Ser. No. 61/793,924, filed Mar.15, 2013, entitled “Voice and Touch User Interface”; U.S. applicationSer. No. 13/032,614, filed Feb. 22, 2011, entitled—Pushing a GraphicalUser Interface to a Remote Device with Display Rules Provided by theRemote Device”; U.S. application Ser. No. 12/683,218, filed Jan. 6,2010, entitled “Pushing a User Interface to a Remote Device”; U.S.application Ser. No. 12/119,960, filed May 13, 2008, entitled “Pushing aUser Interface to a Remote Device”; U.S. application Ser. No.13/175,581, filed Jul. 1, 2011, entitled “Pushing a User Interface to aRemote Device”; U.S. application Ser. No. 13/161,339, filed Jun. 15,2011, entitled “Pushing a Graphical User Interface to a Remote Devicewith Display Rules Provided by the Remote Device”; U.S. application Ser.No. 13/250,947, filed Sep. 30, 2011, entitled “Automatically AdaptingUser Interfaces for Hands-Free Interaction”; U.S. application Ser. No.12/987,982, filed Jan. 10, 2011, entitled “Intelligent AutomatedAssistant”; U.S. Provisional Application Ser. No. 61/295,774, filed Jan.18, 2010, entitled “Intelligent Automated Assistant”; U.S. ProvisionalApplication Ser. No. 61/493,201, filed Jun. 3, 2011, entitled“Generating and Processing Data Items that Represent Tasks to Perform”;U.S. Provisional Application Ser. No. 61/657,744, filed Jun. 9, 2012,entitled “Automatically Adapting User Interface for Hands-FreeInteraction”; U.S. application Ser. No. 12/207,316, filed Sep. 9, 2008,entitled “Radio with Personal DJ”; U.S. Provisional Application Ser. No.61/727,554, filed Nov. 16, 2012, entitled “System and Method forNegotiating Control of a Shared Audio or Visual Resource”; U.S.Application Ser. No. 61/832,818, filed Jun. 8, 2013, entitled “MappingApplication with Several User Interfaces,”; U.S. Provisional ApplicationSer. No. 61/832,841, filed Jun. 8, 2013, entitled “Device and Method forGenerating User Interfaces from a Template,”; U.S. application Ser. No.13/913,428, filed Jun. 8, 2013, entitled “Application Gateway forProviding Different User Interfaces for Limited Distraction andNon-Limited Distraction Contexts,” which applications are incorporatedherein by reference in their entireties.

TECHNICAL FIELD

This relates generally to electronic devices, including but not limitedto electronic devices that communicate wirelessly with peripheralelectronic devices.

BACKGROUND

Users require convenient access to information stored on or accessedthrough their portable electronic devices in a variety of settings,including the home, the workplace, and in the car. The use of peripheralaccessories such as peripheral display units can increase the ease ofaccess to information stored on or accessible through portableelectronic devices. For example, users operating motor vehicles may beunable to directly manipulate their portable electronic devices becauseit is inconvenient, unsafe, or illegal. Accordingly, peripheral displayunits and interfaces are required.

SUMMARY

Some solutions for connecting portable electronic devices withperipheral display units or peripheral accessories may requirecumbersome processes for setting up a connection, including physicallyconnecting the device and the peripheral/accessory and/or manuallyproviding authentication data. Furthermore, reconnecting may be aninconvenient and slow process that requires the user to manually accesshis portable electronic device. Additionally, some solutions forconnections with peripherals/accessories, especially wirelessconnections, are battery-intensive and can cause a user to inadvertentlyexhaust the battery of the portable electronic device. Finally, somesolutions may provide a rigid, inflexible user interface on peripheraldisplay units that is uniform across all peripherals or is difficult toreconfigure, or may only be configured through the cumbersome interfaceof the peripheral, if at all. Additionally, some solutions take too longand drain battery unnecessarily.

Accordingly, there is a need for improved methods, devices, andinterfaces for easily and quickly establishing a relationship with anaccessory/peripheral, and for conveniently reconnecting to theperipheral. Such methods, devices, and interfaces optionally complimentconventional methods for establishing relationships between a portabledevice and an accessory/peripheral display unit and for reconnecting tosaid accessory/peripheral in the future. Such methods, devices, andinterfaces reduce the cognitive burden on a user and produce a moreefficient human-machine interface. In addition, these methods, devices,and interfaces save time and thereby preserve energy, which is ofparticular importance in battery-operated devices. For peripherals andaccessories integrated with motor vehicles, reducing the cognitiveburden on a user also improves driver safety.

There is also a need for methods, devices, and interfaces forefficiently and conveniently accessing and understanding informationabout battery usage, particularly but not exclusively when connected viaa connection with peripheral display units. These methods, devices, andinterfaces facilitate the ability to access information about thebattery-life implications of wireless connections with peripheraldisplay units, including as such relationships relate to predicted useractivity such as traveling to a destination. Such methods, devices, andinterfaces reduce the cognitive burden on a user and produce a moreefficient human-machine interface. In addition, these methods, devices,and interfaces save time and thereby preserve energy, which is ofparticular importance in battery-operated devices. Furthermore, thesemethods, devices, and interfaces improve and prolong device functioningby proactively encouraging users to expend battery life wisely and totake actions to preserve battery life.

There is also a need for methods, devices, and interfaces forefficiently and conveniently configuring the user interfaces of one ormore accessories or peripherals, including configuring variousperipheral interfaces independently of one another, configuringperipheral interfaces through the device (thereby bypassing thecumbersome interface of the peripheral itself), and configuringperipheral interfaces at a time when the device is not connected to theperipheral. Such methods, devices, and interfaces reduce the cognitiveburden on a user and produce a more efficient human-machine interface.In addition, these methods, devices, and interfaces save time andthereby preserve energy, which is of particular importance inbattery-operated devices. For peripherals and accessories integratedwith motor vehicles, reducing the cognitive burden on a user alsoimproves driver safety.

The above deficiencies and other problems are reduced or eliminated bythe disclosed devices, methods, and computer-readable media. In someembodiments, the device is a desktop computer. In some embodiments, thedevice is portable (e.g., a notebook computer, tablet computer, orhandheld device). In some embodiments, the device has a touchpad. Insome embodiments, the device has a touch-sensitive display (also knownas a “touch screen” or “touch screen display”). In some embodiments, thedevice has a graphical user interface (GUI), one or more processors,memory, and one or more modules, programs, or sets of instructionsstored in the memory for performing multiple functions. In someembodiments, the user interacts with the GUI primarily through fingercontacts and gestures on the touch-sensitive surface. In someembodiments, the functions optionally include image editing, drawing,presenting, word processing, website creating, disk authoring,spreadsheet making, game playing, telephoning, video conferencing,e-mailing, instant messaging, workout support, digital photographing,digital videoing, web browsing, digital music playing, and/or digitalvideo playing. Executable instructions for performing these functionsare, optionally, included in a non-transitory computer-readable storagemedium or other computer program product configured for execution by oneor more processors. Executable instructions for performing thesefunctions are, optionally, included in a transitory computer-readablestorage medium or other computer program product configured forexecution by one or more processors.

In some embodiments, at an electronic device with one or moreprocessors, a method is performed, comprising: receiving a request froma user to authorize a relationship that corresponds to a connectionbetween the device and the peripheral display unit, wherein theconnection is a connection over a first data connection with theperipheral display unit; in response to receiving the request toauthorize a relationship, establishing a relationship with theperipheral display unit, wherein establishing the relationship includesreceiving authentication information from the peripheral display unitvia a second data connection that is different from the first dataconnection; while a connection between the device and the peripheraldisplay unit over the first data connection is not active: detectingthat the peripheral display unit is available for establishment of aconnection; and in response to detecting that the peripheral displayunit is available for establishment of a connection, establishing aconnection between the device and the peripheral display unit, whereinthe connection is a connection over the first data connection, andwherein establishing the connection between the device and theperipheral display unit comprises providing the authenticationinformation to the peripheral display unit to establish the connection.

In some embodiments, at an electronic device with one or more processorsand memory, a method is performed, comprising: monitoring battery usageof the device, wherein monitoring battery usage of the device includesmonitoring a charge level of one or more batteries of the device andmonitoring battery usage patterns of the one or more batteries of thedevice; while monitoring battery usage of the device: in accordance witha determination that a charge level of the device meets charge-levelnotification criteria, providing a charge-level alert that indicates acurrent charge level of the one or more batteries; and in accordancewith a determination that battery usage of the device meetsbattery-usage notification criteria different from the charge-levelnotification criteria, providing a battery-usage alert that indicates acurrent battery usage pattern.

In some embodiments, at a portable electronic device having a displayand a communication interface, a method is performed, comprising:displaying on the display of the device a configuration interface,wherein the configuration interface comprises a representation of a userinterface of a peripheral display unit; detecting a request to configurethe user interface of the peripheral display unit; in response todetecting the request to configure the user interface, displaying on thedisplay of the device an updated representation of the user interface,wherein the updated representation is generated in accordance with thedetected request; and after detecting the request to configure the userinterface of the peripheral display unit, transmitting instructions tothe peripheral display unit to display the user interface generated inaccordance with the detected request.

In some embodiments, an electronic device includes a processing unitconfigured to: receive a request from a user to authorize a relationshipthat corresponds to a connection between the device and the peripheraldisplay unit over a first data connection with the peripheral displayunit; and in response to receiving the request to authorize arelationship, establish a relationship with the peripheral display unit,wherein establishing the relationship includes enabling receivingauthentication information from the peripheral display unit via a seconddata connection that is different from the first data connection. Theprocessing unit is further configured to, while a connection between thedevice and the peripheral display unit over the first data connection isnot active: detect that the peripheral display unit is available forestablishment of a connection; and in response to detecting that theperipheral display unit is available for establishment of a connection,establish a connection between the device and the peripheral displayunit, wherein the connection is a connection over the first dataconnection, and wherein establishing the connection between the deviceand the peripheral display unit comprises enabling providing theauthentication information to the peripheral display unit to establishthe connection.

In some embodiments, an electronic device includes a processing unitconfigured to: monitor battery usage of the device, wherein monitoringbattery usage of the device includes monitoring a charge level of one ormore batteries of the device and monitoring battery usage patterns ofthe one or more batteries of the device; and, while monitoring batteryusage of the device, in accordance with a determination that a chargelevel of the device meets charge-level notification criteria, enableproviding a charge-level alert that indicates a current charge level ofthe one or more batteries; and, in accordance with a determination thatbattery usage of the device meets battery-usage notification criteriadifferent from the charge-level notification criteria, enable providinga battery-usage alert that indicates a current battery usage pattern.

In some embodiments, an electronic device includes a display unitconfigured to display a graphical user interface, a communication unitconfigured to send data to a peripheral display unit, and a processingunit configured to: enable displaying on the display unit of the devicea configuration interface, wherein the configuration interface comprisesa representation of a user interface of a peripheral display unit;detect a request to configure the user interface of the peripheraldisplay unit; in response to detecting the request to configure the userinterface, enable displaying on the display unit of the device anupdated representation of the user interface, wherein the updatedrepresentation is generated in accordance with the detected request. Theprocessing unit is further configured to, after detecting the request toconfigure the user interface of the peripheral display unit, enabletransmitting instructions to the peripheral display unit to display theuser interface generated in accordance with the detected request.

In some embodiments, a non-transitory computer readable storage mediumstores one or more programs, the one or more programs comprisinginstructions, which when executed by a portable multifunction device,cause the device to: receive a request from a user to authorize arelationship that corresponds to a connection between the device and aperipheral display unit, wherein the connection is a connection over afirst data connection with the peripheral display unit; in response toreceiving the request to authorize a relationship, establish arelationship with the peripheral display unit, wherein establishing therelationship includes receiving authentication information from theperipheral display unit via a second data connection that is differentfrom the first data connection; and while a connection between thedevice and the peripheral display unit over the first data connection isnot active: detect that the peripheral display unit is available forestablishment of a connection; and in response to detecting that theperipheral display unit is available for establishment of a connection,establish a connection between the device and the peripheral displayunit, wherein the connection is a connection over the first dataconnection, and wherein establishing the connection between the deviceand the peripheral display unit comprises providing the authenticationinformation to the peripheral display unit to establish the connection.

In some embodiments, a non-transitory computer readable storage mediumstores one or more programs, the one or more programs comprisinginstructions, which when executed by a portable multifunction device,cause the device to: monitor battery usage of the device, whereinmonitoring battery usage of the device includes monitoring a chargelevel of one or more batteries of the device and monitoring batteryusage patterns of the one or more batteries of the device; and whilemonitoring battery usage of the device: in accordance with adetermination that a charge level of the device meets charge-levelnotification criteria, provide a charge-level alert that indicates acurrent charge level of the one or more batteries; and in accordancewith a determination that battery usage of the device meetsbattery-usage notification criteria different from the charge-levelnotification criteria, provide a battery-usage alert that indicates acurrent battery usage pattern.

In some embodiments, a non-transitory computer readable storage mediumstores one or more programs, the one or more programs comprisinginstructions, which when executed by a portable multifunction devicewith a display, cause the device to: display on the display of thedevice a configuration interface, wherein the configuration interfacecomprises a representation of a user interface of a peripheral displayunit; detect a request to configure the user interface of the peripheraldisplay unit; in response to detecting the request to configure the userinterface, display on the display of the device an updatedrepresentation of the user interface, wherein the updated representationis generated in accordance with the detected request; and afterdetecting the request to configure the user interface of the peripheraldisplay unit, transmit instructions to the peripheral display unit todisplay the user interface generated in accordance with the detectedrequest.

In some embodiments, a transitory computer readable storage mediumstores one or more programs, the one or more programs comprisinginstructions, which when executed by a portable multifunction device,cause the device to: receive a request from a user to authorize arelationship that corresponds to a connection between the device and aperipheral display unit, wherein the connection is a connection over afirst data connection with the peripheral display unit; in response toreceiving the request to authorize a relationship, establish arelationship with the peripheral display unit, wherein establishing therelationship includes receiving authentication information from theperipheral display unit via a second data connection that is differentfrom the first data connection; and while a connection between thedevice and the peripheral display unit over the first data connection isnot active: detect that the peripheral display unit is available forestablishment of a connection; and in response to detecting that theperipheral display unit is available for establishment of a connection,establish a connection between the device and the peripheral displayunit, wherein the connection is a connection over the first dataconnection, and wherein establishing the connection between the deviceand the peripheral display unit comprises providing the authenticationinformation to the peripheral display unit to establish the connection.

In some embodiments, a transitory computer readable storage mediumstores one or more programs, the one or more programs comprisinginstructions, which when executed by a portable multifunction device,cause the device to: monitor battery usage of the device, whereinmonitoring battery usage of the device includes monitoring a chargelevel of one or more batteries of the device and monitoring batteryusage patterns of the one or more batteries of the device; and whilemonitoring battery usage of the device: in accordance with adetermination that a charge level of the device meets charge-levelnotification criteria, provide a charge-level alert that indicates acurrent charge level of the one or more batteries; and in accordancewith a determination that battery usage of the device meetsbattery-usage notification criteria different from the charge-levelnotification criteria, provide a battery-usage alert that indicates acurrent battery usage pattern.

In some embodiments, a transitory computer readable storage mediumstores one or more programs, the one or more programs comprisinginstructions, which when executed by a portable multifunction devicewith a display, cause the device to: display on the display of thedevice a configuration interface, wherein the configuration interfacecomprises a representation of a user interface of a peripheral displayunit; detect a request to configure the user interface of the peripheraldisplay unit; in response to detecting the request to configure the userinterface, display on the display of the device an updatedrepresentation of the user interface, wherein the updated representationis generated in accordance with the detected request; and afterdetecting the request to configure the user interface of the peripheraldisplay unit, transmit instructions to the peripheral display unit todisplay the user interface generated in accordance with the detectedrequest.

In some embodiments, a device comprises: one or more processors; andmemory storing instructions that, when executed by the one or moreprocessors, cause the device to: receive a request from a user toauthorize a relationship that corresponds to a connection between thedevice and a peripheral display unit, wherein the connection is aconnection over a first data connection with the peripheral displayunit; in response to receiving the request to authorize a relationship,establish a relationship with the peripheral display unit, whereinestablishing the relationship includes receiving authenticationinformation from the peripheral display unit via a second dataconnection that is different from the first data connection; and while aconnection between the device and the peripheral display unit over thefirst data connection is not active: detect that the peripheral displayunit is available for establishment of a connection; and in response todetecting that the peripheral display unit is available forestablishment of a connection, establish a connection between the deviceand the peripheral display unit, wherein the connection is a connectionover the first data connection, and wherein establishing the connectionbetween the device and the peripheral display unit comprises providingthe authentication information to the peripheral display unit toestablish the connection.

In some embodiments, a device comprises: one or more processors; andmemory storing instructions that, when executed by the one or moreprocessors, cause the device to: monitor battery usage of the device,wherein monitoring battery usage of the device includes monitoring acharge level of one or more batteries of the device and monitoringbattery usage patterns of the one or more batteries of the device; andwhile monitoring battery usage of the device: in accordance with adetermination that a charge level of the device meets charge-levelnotification criteria, provide a charge-level alert that indicates acurrent charge level of the one or more batteries; and in accordancewith a determination that battery usage of the device meetsbattery-usage notification criteria different from the charge-levelnotification criteria, provide a battery-usage alert that indicates acurrent battery usage pattern.

In some embodiments, a device comprises: a display; one or moreprocessors; and memory storing instructions that, when executed by theone or more processors, cause the device to: display on the display ofthe device a configuration interface, wherein the configurationinterface comprises a representation of a user interface of a peripheraldisplay unit; detect a request to configure the user interface of theperipheral display unit; in response to detecting the request toconfigure the user interface, display on the display of the device anupdated representation of the user interface, wherein the updatedrepresentation is generated in accordance with the detected request; andafter detecting the request to configure the user interface of theperipheral display unit, transmit instructions to the peripheral displayunit to display the user interface generated in accordance with thedetected request.

In some embodiments, a device comprising: means for receiving a requestfrom a user to authorize a relationship that corresponds to a connectionbetween the device and a peripheral display unit, wherein the connectionis a connection over a first data connection with the peripheral displayunit; means for, in response to receiving the request to authorize arelationship, establishing a relationship with the peripheral displayunit, wherein establishing the relationship includes receivingauthentication information from the peripheral display unit via a seconddata connection that is different from the first data connection; andmeans for, while a connection between the device and the peripheraldisplay unit over the first data connection is not active: detectingthat the peripheral display unit is available for establishment of aconnection; and in response to detecting that the peripheral displayunit is available for establishment of a connection, establishing aconnection between the device and the peripheral display unit, whereinthe connection is a connection over the first data connection, andwherein establishing the connection between the device and theperipheral display unit comprises providing the authenticationinformation to the peripheral display unit to establish the connection.

In some embodiments, a device comprises; means for monitoring batteryusage of the device, wherein monitoring battery usage of the deviceincludes monitoring a charge level of one or more batteries of thedevice and monitoring battery usage patterns of the one or morebatteries of the device; and means for, while monitoring battery usageof the device: in accordance with a determination that a charge level ofthe device meets charge-level notification criteria, providing acharge-level alert that indicates a current charge level of the one ormore batteries; and in accordance with a determination that batteryusage of the device meets battery-usage notification criteria differentfrom the charge-level notification criteria, providing a battery-usagealert that indicates a current battery usage pattern.

In some embodiments, a device comprises: means for displaying on thedisplay of the device a configuration interface, wherein theconfiguration interface comprises a representation of a user interfaceof a peripheral display unit; means for detecting a request to configurethe user interface of the peripheral display unit; means for, inresponse to detecting the request to configure the user interface,displaying on the display of the device an updated representation of theuser interface, wherein the updated representation is generated inaccordance with the detected request; and means for, after detecting therequest to configure the user interface of the peripheral display unit,transmitting instructions to the peripheral display unit to display theuser interface generated in accordance with the detected request.

Thus, in some embodiments, electronic devices with displays are providedwith more efficient methods and interfaces for establishing andoperating a relationship and connection between an electronic device anda peripheral display unit, for monitoring battery usage patterns ofconnected devices, and for configuring user interfaces of peripheraldisplay units. The effectiveness, efficiency, and user satisfaction withsuch devices may thereby be increased. Such methods and interfaces canoptionally complement or replace conventional methods.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the various described embodiments,reference should be made to the Description of Embodiments below, inconjunction with the following drawings in which like reference numeralsrefer to corresponding parts throughout the figures.

FIG. 1A is a block diagram illustrating a portable multifunction devicewith a touch-sensitive display in accordance with some embodiments.

FIG. 1B is a block diagram illustrating exemplary components for eventhandling in accordance with some embodiments.

FIG. 2 illustrates a portable multifunction device having a touch screenin accordance with some embodiments.

FIG. 3 is a block diagram of an exemplary multifunction device with adisplay and a touch-sensitive surface in accordance with someembodiments.

FIG. 4A illustrates an exemplary user interface for a menu ofapplications on a portable multifunction device in accordance with someembodiments.

FIG. 4B illustrates an exemplary user interface for a multifunctiondevice with a touch-sensitive surface that is separate from the displayin accordance with some embodiments.

FIG. 5A is a block diagram illustrating an operating environment inwhich a portable multifunction device communicates with an externalpresentation system (e.g., peripheral display unit) and/or server inaccordance with some embodiments.

FIG. 5B is a flow diagram illustrating a method of sending updateinformation to an affected display in accordance with some embodiments.

FIGS. 6A-6I illustrate user interfaces for establishing and operating awireless data connection between a device and a peripheral display unitin accordance with some embodiments.

FIGS. 6J-6M illustrate user interfaces monitoring battery-usage patternsand providing battery-usage alerts in accordance with some embodiments.

FIGS. 6N-6W illustrate user interfaces for configuring a user interfaceof a peripheral display unit in accordance with some embodiments.

FIGS. 7A-7H are flow diagrams illustrating methods of establishing andoperating a data connection between a device and a peripheral displayunit in accordance with some embodiments.

FIGS. 8A-8D are flow diagrams illustrating methods of monitoringbattery-usage patterns and providing battery-usage alerts in accordancewith some embodiments.

FIGS. 9A-9E are flow diagrams illustrating methods of configuring a userinterface of a peripheral display unit in accordance with someembodiments.

FIG. 10 is a functional block diagram of an electronic device inaccordance with some embodiments.

FIG. 11 is a functional block diagram of an electronic device inaccordance with some embodiments.

FIG. 12 is a functional block diagram of an electronic device inaccordance with some embodiments.

DESCRIPTION OF EMBODIMENTS

There is a need for improved devices, methods, and computer-readablemedia for establishing and operating a connection between an electronicdevice and a peripheral display unit. The embodiments described hereinimprove on current methods by allowing for efficient, convenient, fast,and intuitive ways to establish a relationship between a device and aperipheral display unit, establish and reestablish a data connectionbetween a device and a peripheral display unit, display charge levelnotifications and battery-usage notifications that are useful when adevice is connected with a peripheral display unit, and configure theinterface of a peripheral display unit from a device, among otherfunctions and features.

Below, FIGS. 1A-1B, 2, and 3 provide a description of exemplary devices.FIGS. 4A-4B illustrate exemplary user interfaces. FIG. 5A illustrates anexemplary operating environment. FIG. 5B illustrates a flow diagramillustrating an exemplary method. FIGS. 6A-6W illustrate exemplary userinterfaces. FIGS. 7A-7H, 8A-8D, and 9A-9E are flow diagrams illustratingexemplary methods. FIGS. 10, 11, and 12 are a functional block diagramsillustrating exemplary devices. The user interfaces in FIGS. 6A-6W areused to illustrate the processes in FIGS. 7A-7H, 8A-8D, and 9A-9E.

Exemplary Devices

Reference will now be made in detail to embodiments, examples of whichare illustrated in the accompanying drawings. In the following detaileddescription, numerous specific details are set forth in order to providea thorough understanding of the various described embodiments. However,it will be apparent to one of ordinary skill in the art that the variousdescribed embodiments can optionally be practiced without these specificdetails. In other instances, well-known methods, procedures, components,circuits, and networks have not been described in detail so as not tounnecessarily obscure aspects of the embodiments.

It will also be understood that, although the terms first, second, etc.are, in some instances, used herein to describe various elements, theseelements should not be limited by these terms. These terms are only usedto distinguish one element from another. For example, a first contactcould be termed a second contact, and, similarly, a second contact couldbe termed a first contact, without departing from the scope of thevarious described embodiments. The first contact and the second contactare both contacts, but they are not the same contact.

The terminology used in the description of the various describedembodiments herein is for the purpose of describing particularembodiments only and is not intended to be limiting. As used in thedescription of the various described embodiments and the appendedclaims, the singular forms “a,” “an” and “the” are intended to includethe plural forms as well, unless the context clearly indicatesotherwise. It will also be understood that the term “and/or” as usedherein refers to and encompasses any and all possible combinations ofone or more of the associated listed items. It will be furtherunderstood that the terms “includes,” “including,” “comprises,” and/or“comprising,” when used in this specification, specify the presence ofstated features, integers, steps, operations, elements, and/orcomponents, but do not preclude the presence or addition of one or moreother features, integers, steps, operations, elements, components,and/or groups thereof.

As used herein, the term “if” is, optionally, construed to mean “when”or “upon” or “in response to determining” or “in response to detecting,”depending on the context. Similarly, the phrase “if it is determined” or“if [a stated condition or event] is detected” is, optionally, construedto mean “upon determining” or “in response to determining” or ‘upondetecting [the stated condition or event]” or “in response to detecting[the stated condition or event],” depending on the context.

Embodiments of electronic devices, user interfaces for such devices, andassociated processes for using such devices are described. In someembodiments, the device is a portable communications device, such as amobile telephone, that also contains other functions, such as PDA and/ormusic player functions. Examples of portable multifunction devicesinclude, without limitation, the iPhone®, iPod Touch®, and iPad® devicesfrom Apple Inc. of Cupertino, Calif. Other portable electronic devices,such as laptops or tablet computers with touch-sensitive surfaces (e.g.,touch screen displays and/or touch pads), are, optionally, used. Itshould also be understood that, in some embodiments, the device is not aportable communications device, but is a desktop computer with atouch-sensitive surface (e.g., a touch screen display and/or a touchpad).

In the discussion that follows, an electronic device that includes adisplay and a touch-sensitive surface is described. It should beunderstood, however, that the electronic device optionally includes oneor more other physical user-interface devices, such as a physicalkeyboard, a mouse and/or a joystick.

The device typically supports a variety of applications, such as one ormore of the following: a drawing application, a presentationapplication, a word processing application, a website creationapplication, a disk authoring application, a spreadsheet application, agaming application, a telephone application, a video conferencingapplication, an e-mail application, an instant messaging application, aworkout support application, a photo management application, a digitalcamera application, a digital video camera application, a web browsingapplication, a digital music player application, and/or a digital videoplayer application.

The various applications that are executed on the device optionally useat least one common physical user-interface device, such as thetouch-sensitive surface. One or more functions of the touch-sensitivesurface as well as corresponding information displayed on the deviceare, optionally, adjusted and/or varied from one application to the nextand/or within a respective application. In this way, a common physicalarchitecture (such as the touch-sensitive surface) of the deviceoptionally supports the variety of applications with user interfacesthat are intuitive and transparent to the user.

Attention is now directed toward embodiments of portable devices withtouch-sensitive displays. FIG. 1A is a block diagram illustratingportable multifunction device 100 with touch-sensitive displays 112 inaccordance with some embodiments. Touch-sensitive display 112 issometimes called a “touch screen” for convenience, and is sometimesknown as or called a touch-sensitive display system. Device 100 includesmemory 102 (which optionally includes one or more computer-readablestorage mediums), memory controller 122, one or more processing units(CPU's) 120, peripherals interface 118, RF circuitry 108, audiocircuitry 110, speaker 111, microphone 113, input/output (I/O) subsystem106, other input or control devices 116, and external port 124. Device100 optionally includes one or more optical sensors 164. Device 100optionally includes one or more intensity sensors 165 for detectingintensity of contacts on device 100 (e.g., a touch-sensitive surfacesuch as touch-sensitive display system 112 of device 100). Device 100optionally includes one or more tactile output generators 167 forgenerating tactile outputs on device 100 (e.g., generating tactileoutputs on a touch-sensitive surface such as touch-sensitive displaysystem 112 of device 100 or touchpad 355 of device 300). Thesecomponents optionally communicate over one or more communication busesor signal lines 103.

As used in the specification and claims, the term “intensity” of acontact on a touch-sensitive surface refers to the force or pressure(force per unit area) of a contact (e.g., a finger contact) on the touchsensitive surface, or to a substitute (proxy) for the force or pressureof a contact on the touch sensitive surface. The intensity of a contacthas a range of values that includes at least four distinct values andmore typically includes hundreds of distinct values (e.g., at least256). Intensity of a contact is, optionally, determined (or measured)using various approaches and various sensors or combinations of sensors.For example, one or more force sensors underneath or adjacent to thetouch-sensitive surface are, optionally, used to measure force atvarious points on the touch-sensitive surface. In some implementations,force measurements from multiple force sensors are combined (e.g., aweighted average) to determine an estimated force of a contact.Similarly, a pressure-sensitive tip of a stylus is, optionally, used todetermine a pressure of the stylus on the touch-sensitive surface.Alternatively, the size of the contact area detected on thetouch-sensitive surface and/or changes thereto, the capacitance of thetouch-sensitive surface proximate to the contact and/or changes thereto,and/or the resistance of the touch-sensitive surface proximate to thecontact and/or changes thereto are, optionally, used as a substitute forthe force or pressure of the contact on the touch-sensitive surface. Insome implementations, the substitute measurements for contact force orpressure are used directly to determine whether an intensity thresholdhas been exceeded (e.g., the intensity threshold is described in unitscorresponding to the substitute measurements). In some implementations,the substitute measurements for contact force or pressure are convertedto an estimated force or pressure and the estimated force or pressure isused to determine whether an intensity threshold has been exceeded(e.g., the intensity threshold is a pressure threshold measured in unitsof pressure).

As used in the specification and claims, the term “tactile output”refers to physical displacement of a device relative to a previousposition of the device, physical displacement of a component (e.g., atouch-sensitive surface) of a device relative to another component(e.g., housing) of the device, or displacement of the component relativeto a center of mass of the device that will be detected by a user withthe user's sense of touch. For example, in situations where the deviceor the component of the device is in contact with a surface of a userthat is sensitive to touch (e.g., a finger, palm, or other part of auser's hand), the tactile output generated by the physical displacementwill be interpreted by the user as a tactile sensation corresponding toa perceived change in physical characteristics of the device or thecomponent of the device. For example, movement of a touch-sensitivesurface (e.g., a touch-sensitive display or trackpad) is, optionally,interpreted by the user as a “down click” or “up click” of a physicalactuator button. In some cases, a user will feel a tactile sensationsuch as an “down click” or “up click” even when there is no movement ofa physical actuator button associated with the touch-sensitive surfacethat is physically pressed (e.g., displaced) by the user's movements. Asanother example, movement of the touch-sensitive surface is, optionally,interpreted or sensed by the user as “roughness” of the touch-sensitivesurface, even when there is no change in smoothness of thetouch-sensitive surface. While such interpretations of touch by a userwill be subject to the individualized sensory perceptions of the user,there are many sensory perceptions of touch that are common to a largemajority of users. Thus, when a tactile output is described ascorresponding to a particular sensory perception of a user (e.g., an “upclick,” a “down click,” “roughness”), unless otherwise stated, thegenerated tactile output corresponds to physical displacement of thedevice or a component thereof that will generate the described sensoryperception for a typical (or average) user.

It should be appreciated that device 100 is only one example of aportable multifunction device, and that device 100 optionally has moreor fewer components than shown, optionally combines two or morecomponents, or optionally has a different configuration or arrangementof the components. The various components shown in FIG. 1A areimplemented in hardware, software, or a combination of both hardware andsoftware, including one or more signal processing and/or applicationspecific integrated circuits.

Memory 102 optionally includes high-speed random access memory andoptionally also includes non-volatile memory, such as one or moremagnetic disk storage devices, flash memory devices, or othernon-volatile solid-state memory devices. Access to memory 102 by othercomponents of device 100, such as CPU 120 and the peripherals interface118, is, optionally, controlled by memory controller 122.

Peripherals interface 118 can be used to couple input and outputperipherals of the device to CPU 120 and memory 102. The one or moreprocessors 120 run or execute various software programs and/or sets ofinstructions stored in memory 102 to perform various functions fordevice 100 and to process data.

In some embodiments, peripherals interface 118, CPU 120, and memorycontroller 122 are, optionally, implemented on a single chip, such aschip 104. In some other embodiments, they are, optionally, implementedon separate chips.

RF (radio frequency) circuitry 108 receives and sends RF signals, alsocalled electromagnetic signals. RF circuitry 108 converts electricalsignals to/from electromagnetic signals and communicates withcommunications networks and other communications devices via theelectromagnetic signals. RF circuitry 108 optionally includes well-knowncircuitry for performing these functions, including but not limited toan antenna system, an RF transceiver, one or more amplifiers, a tuner,one or more oscillators, a digital signal processor, a CODEC chipset, asubscriber identity module (SIM) card, memory, and so forth. RFcircuitry 108 optionally communicates with networks, such as theInternet, also referred to as the World Wide Web (WWW), an intranetand/or a wireless network, such as a cellular telephone network, awireless local area network (LAN) and/or a metropolitan area network(MAN), and other devices by wireless communication. The wirelesscommunication optionally uses any of a plurality of communicationsstandards, protocols and technologies, including but not limited toGlobal System for Mobile Communications (GSM), Enhanced Data GSMEnvironment (EDGE), high-speed downlink packet access (HSDPA),high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO),HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), long term evolution (LTE), nearfield communication (NFC), wideband code division multiple access(W-CDMA), code division multiple access (CDMA), time division multipleaccess (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a,IEEE 802.11b, IEEE 802.11g and/or IEEE 802.1 in), voice over InternetProtocol (VoIP), Wi-MAX, a protocol for e-mail (e.g., Internet messageaccess protocol (IMAP) and/or post office protocol (POP)), instantmessaging (e.g., extensible messaging and presence protocol (XMPP),Session Initiation Protocol for Instant Messaging and PresenceLeveraging Extensions (SIMPLE), Instant Messaging and Presence Service(IMPS)), and/or Short Message Service (SMS), or any other suitablecommunication protocol, including communication protocols not yetdeveloped as of the filing date of this document.

Audio circuitry 110, speaker 111, and microphone 113 provide an audiointerface between a user and device 100. Audio circuitry 110 receivesaudio data from peripherals interface 118, converts the audio data to anelectrical signal, and transmits the electrical signal to speaker 111.Speaker 11 converts the electrical signal to human-audible sound waves.Audio circuitry 110 also receives electrical signals converted bymicrophone 113 from sound waves. Audio circuitry 110 converts theelectrical signal to audio data and transmits the audio data toperipherals interface 118 for processing. Audio data is, optionally,retrieved from and/or transmitted to memory 102 and/or RF circuitry 108by peripherals interface 118. In some embodiments, audio circuitry 110also includes a headset jack (e.g., 212, FIG. 2). The headset jackprovides an interface between audio circuitry 110 and removable audioinput/output peripherals, such as output-only headphones or a headsetwith both output (e.g., a headphone for one or both ears) and input(e.g., a microphone).

I/O subsystem 106 couples input/output peripherals on device 100, suchas touch screen 112 and other input control devices 116, to peripheralsinterface 118. I/O subsystem 106 optionally includes display controller156, optical sensor controller 158, intensity sensor controller 159,haptic feedback controller 161 and one or more input controllers 160 forother input or control devices. The one or more input controllers 160receive/send electrical signals from/to other input or control devices116. The other input control devices 116 optionally include physicalbuttons (e.g., push buttons, rocker buttons, etc.), dials, sliderswitches, joysticks, click wheels, and so forth. In some alternateembodiments, input controller(s) 160 are, optionally, coupled to any (ornone) of the following: a keyboard, infrared port, USB port, and apointer device such as a mouse. The one or more buttons (e.g., 208, FIG.2) optionally include an up/down button for volume control of speaker111 and/or microphone 113. The one or more buttons optionally include apush button (e.g., 206, FIG. 2).

Touch-sensitive display 112 provides an input interface and an outputinterface between the device and a user. Display controller 156 receivesand/or sends electrical signals from/to touch screen 112. Touch screen112 displays visual output to the user. The visual output optionallyincludes graphics, text, icons, video, and any combination thereof(collectively termed “graphics”). In some embodiments, some or all ofthe visual output corresponds to user-interface objects.

Touch screen 112 has a touch-sensitive surface, sensor or set of sensorsthat accepts input from the user based on haptic and/or tactile contact.Touch screen 112 and display controller 156 (along with any associatedmodules and/or sets of instructions in memory 102) detect contact (andany movement or breaking of the contact) on touch screen 112 andconverts the detected contact into interaction with user-interfaceobjects (e.g., one or more soft keys, icons, web pages or images) thatare displayed on touch screen 112. In an example, a point of contactbetween touch screen 112 and the user corresponds to a finger of theuser.

Touch screen 112 optionally uses LCD (liquid crystal display)technology, LPD (light emitting polymer display) technology, or LED(light emitting diode) technology, although other display technologiesare used in other embodiments. Touch screen 112 and display controller156 optionally detect contact and any movement or breaking thereof usingany of a plurality of touch sensing technologies now known or laterdeveloped, including but not limited to capacitive, resistive, infrared,and surface acoustic wave technologies, as well as other proximitysensor arrays or other elements for determining one or more points ofcontact with touch screen 112. In an example, projected mutualcapacitance sensing technology is used, such as that found in theiPhone®, iPod Touch®, and iPad® from Apple Inc. of Cupertino, Calif.

Touch screen 112 optionally has a video resolution in excess of 100 dpi.In some embodiments, the touch screen has a video resolution ofapproximately 160 dpi. The user optionally makes contact with touchscreen 112 using any suitable object or appendage, such as a stylus, afinger, and so forth. In some embodiments, the user interface isdesigned to work primarily with finger-based contacts and gestures,which can be less precise than stylus-based input due to the larger areaof contact of a finger on the touch screen. In some embodiments, thedevice translates the rough finger-based input into a precisepointer/cursor position or command for performing the actions desired bythe user.

In some embodiments, in addition to the touch screen, device 100optionally includes a touchpad (not shown) for activating ordeactivating particular functions. In some embodiments, the touchpad isa touch-sensitive area of the device that, unlike the touch screen, doesnot display visual output. The touchpad is, optionally, atouch-sensitive surface that is separate from touch screen 112 or anextension of the touch-sensitive surface formed by the touch screen.

Device 100 also includes power system 162 for powering the variouscomponents. Power system 162 optionally includes a power managementsystem, one or more power sources (e.g., battery, alternating current(AC)), a recharging system, a power failure detection circuit, a powerconverter or inverter, a power status indicator (e.g., a light-emittingdiode (LED)) and any other components associated with the generation,management and distribution of power in portable devices.

Device 100 optionally also includes one or more optical sensors 164.FIG. 1A shows an optical sensor coupled to optical sensor controller 158in I/O subsystem 106. Optical sensor 164 optionally includescharge-coupled device (CCD) or complementary metal-oxide semiconductor(CMOS) phototransistors. Optical sensor 164 receives light from theenvironment, projected through one or more lens, and converts the lightto data representing an image. In conjunction with imaging module 143(also called a camera module), optical sensor 164 optionally capturesstill images or video. In some embodiments, an optical sensor is locatedon the back of device 100, opposite touch screen display 112 on thefront of the device, so that the touch screen display is enabled for useas a viewfinder for still and/or video image acquisition. In someembodiments, another optical sensor is located on the front of thedevice so that the user's image is, optionally, obtained forvideoconferencing while the user views the other video conferenceparticipants on the touch screen display.

Device 100 optionally also includes one or more contact intensitysensors 165. FIG. 1A shows a contact intensity sensor coupled tointensity sensor controller 159 in I/O subsystem 106. Contact intensitysensor 165 optionally includes one or more piezoresistive strain gauges,capacitive force sensors, electric force sensors, piezoelectric forcesensors, optical force sensors, capacitive touch-sensitive surfaces, orother intensity sensors (e.g., sensors used to measure the force (orpressure) of a contact on a touch-sensitive surface). Contact intensitysensor 165 receives contact intensity information (e.g., pressureinformation or a proxy for pressure information) from the environment.In some embodiments, at least one contact intensity sensor is collocatedwith, or proximate to, a touch-sensitive surface (e.g., touch-sensitivedisplay system 112). In some embodiments, at least one contact intensitysensor is located on the back of device 100, opposite touch screendisplay 112 which is located on the front of device 100.

Device 100 optionally also includes one or more proximity sensors 166.FIG. 1A shows proximity sensor 166 coupled to peripherals interface 118.Alternately, proximity sensor 166 is coupled to input controller 160 inI/O subsystem 106. In some embodiments, the proximity sensor turns offand disables touch screen 112 when the multifunction device is placednear the user's ear (e.g., when the user is making a phone call).

Device 100 optionally also includes one or more tactile outputgenerators 167. FIG. 1A shows a tactile output generator coupled tohaptic feedback controller 161 in I/O subsystem 106. Tactile outputgenerator 167 optionally includes one or more electroacoustic devicessuch as speakers or other audio components and/or electromechanicaldevices that convert energy into linear motion such as a motor,solenoid, electroactive polymer, piezoelectric actuator, electrostaticactuator, or other tactile output generating component (e.g., acomponent that converts electrical signals into tactile outputs on thedevice). Contact intensity sensor 165 receives tactile feedbackgeneration instructions from haptic feedback module 133 and generatestactile outputs on device 100 that are capable of being sensed by a userof device 100. In some embodiments, at least one tactile outputgenerator is collocated with, or proximate to, a touch-sensitive surface(e.g., touch-sensitive display system 112) and, optionally, generates atactile output by moving the touch-sensitive surface vertically (e.g.,in/out of a surface of device 100) or laterally (e.g., back and forth inthe same plane as a surface of device 100). In some embodiments, atleast one tactile output generator sensor is located on the back ofdevice 100, opposite touch screen display 112 which is located on thefront of device 100.

Device 100 optionally also includes one or more accelerometers 168. FIG.1A shows accelerometer 168 coupled to peripherals interface 118.Alternately, accelerometer 168 is, optionally, coupled to an inputcontroller 160 in I/O subsystem 106. In some embodiments, information isdisplayed on the touch screen display in a portrait view or a landscapeview based on an analysis of data received from the one or moreaccelerometers. Device 100 optionally includes, in addition toaccelerometer(s) 168, a magnetometer (not shown) and a GPS (or GLONASSor other global navigation system) receiver (not shown) for obtaininginformation concerning the location and orientation (e.g., portrait orlandscape) of device 100.

In some embodiments, the software components stored in memory 102include operating system 126, communication module (or set ofinstructions) 128, contact/motion module (or set of instructions) 130,graphics module (or set of instructions) 132, text input module (or setof instructions) 134, Global Positioning System (GPS) module (or set ofinstructions) 135, and applications (or sets of instructions) 136.Furthermore, in some embodiments memory 102 stores device/globalinternal state 157, as shown in FIGS. 1A and 3. Device/global internalstate 157 includes one or more of: active application state, indicatingwhich applications, if any, are currently active; display state,indicating what applications, views or other information occupy variousregions of touch screen display 112; sensor state, including informationobtained from the device's various sensors and input control devices116; and location information concerning the device's location and/orattitude.

Operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, oran embedded operating system such as VxWorks) includes various softwarecomponents and/or drivers for controlling and managing general systemtasks (e.g., memory management, storage device control, powermanagement, etc.) and facilitates communication between various hardwareand software components.

Communication module 128 facilitates communication with other devicesover one or more external ports 124 and also includes various softwarecomponents for handling data received by RF circuitry 108 and/orexternal port 124. External port 124 (e.g., Universal Serial Bus (USB),FIREWIRE, etc.) is adapted for coupling directly to other devices orindirectly over a network (e.g., the Internet, wireless LAN, etc.). Insome embodiments, the external port is a multi-pin (e.g., 30-pin)connector that is the same as, or similar to and/or compatible with the30-pin connector used on iPod (trademark of Apple Inc.) devices. In someembodiments, the external port is a multi-pin (e.g., 8-pin) connectorthat is the same as, or similar to and/or compatible with the 8-pinconnector (e.g., Lightning connector) used on iPhone and iPod (trademarkof Apple Inc.) devices.

Contact/motion module 130 optionally detects contact with touch screen112 (in conjunction with display controller 156) and other touchsensitive devices (e.g., a touchpad or physical click wheel).Contact/motion module 130 includes various software components forperforming various operations related to detection of contact, such asdetermining if contact has occurred (e.g., detecting a finger-downevent), determining an intensity of the contact (e.g., the force orpressure of the contact or a substitute for the force or pressure of thecontact), determining if there is movement of the contact and trackingthe movement across the touch-sensitive surface (e.g., detecting one ormore finger-dragging events), and determining if the contact has ceased(e.g., detecting a finger-up event or a break in contact).Contact/motion module 130 receives contact data from the touch-sensitivesurface. Determining movement of the point of contact, which isrepresented by a series of contact data, optionally includes determiningspeed (magnitude), velocity (magnitude and direction), and/or anacceleration (a change in magnitude and/or direction) of the point ofcontact. These operations are, optionally, applied to single contacts(e.g., one finger contacts) or to multiple simultaneous contacts (e.g.,“multitouch”/multiple finger contacts). In some embodiments,contact/motion module 130 and display controller 156 detect contact on atouchpad.

In some embodiments, contact/motion module 130 uses a set of one or moreintensity thresholds to determine whether an operation has beenperformed by a user (e.g., to determine whether a user has “clicked” onan icon). In some embodiments at least a subset of the intensitythresholds are determined in accordance with software parameters (e.g.,the intensity thresholds are not determined by the activation thresholdsof particular physical actuators and can be adjusted without changingthe physical hardware of device 100). For example, a mouse “click”threshold of a trackpad or touch screen display can be set to any of alarge range of predefined thresholds values without changing thetrackpad or touch screen display hardware. Additionally, in someimplementations a user of the device is provided with software settingsfor adjusting one or more of the set of intensity thresholds (e.g., byadjusting individual intensity thresholds and/or by adjusting aplurality of intensity thresholds at once with a system-level click“intensity” parameter).

Contact/motion module 130 optionally detects a gesture input by a user.Different gestures on the touch-sensitive surface have different contactpatterns (e.g., different motions, timings, and/or intensities ofdetected contacts). Thus, a gesture is, optionally, detected bydetecting a particular contact pattern. For example, detecting a fingertap gesture includes detecting a finger-down event followed by detectinga finger-up (lift off) event at the same position (or substantially thesame position) as the finger-down event (e.g., at the position of anicon). As another example, detecting a finger swipe gesture on thetouch-sensitive surface includes detecting a finger-down event followedby detecting one or more finger-dragging events, and subsequentlyfollowed by detecting a finger-up (lift off) event.

Graphics module 132 includes various known software components forrendering and displaying graphics on touch screen 112 or other display,including components for changing the visual impact (e.g., brightness,transparency, saturation, contrast or other visual property) of graphicsthat are displayed. As used herein, the term “graphics” includes anyobject that can be displayed to a user, including without limitationtext, web pages, icons (such as user-interface objects including softkeys), digital images, videos, animations and the like.

In some embodiments, graphics module 132 stores data representinggraphics to be used. Each graphic is, optionally, assigned acorresponding code. Graphics module 132 receives, from applicationsetc., one or more codes specifying graphics to be displayed along with,if necessary, coordinate data and other graphic property data, and thengenerates screen image data to output to display controller 156.

Haptic feedback module 133 includes various software components forgenerating instructions used by tactile output generator(s) 167 toproduce tactile outputs at one or more locations on device 100 inresponse to user interactions with device 100.

Text input module 134, which is, optionally, a component of graphicsmodule 132, provides soft keyboards for entering text in variousapplications (e.g., contacts 137, e-mail 140, IM 141, browser 147, andany other application that needs text input).

GPS module 135 determines the location of the device and provides thisinformation for use in various applications (e.g., to telephone 138 foruse in location-based dialing, to camera 143 as picture/video metadata,and to applications that provide location-based services such as weatherwidgets, local yellow page widgets, and map/navigation widgets).

Applications 136 optionally include the following modules (or sets ofinstructions), or a subset or superset thereof:

-   -   contacts module 137 (sometimes called an address book or contact        list);    -   telephone module 138;    -   video conferencing module 139;    -   e-mail client module 140;    -   instant messaging (IM) module 141;    -   workout support module 142;    -   camera module 143 for still and/or video images;    -   image management module 144;    -   browser module 147;    -   calendar module 148;    -   widget modules 149, which optionally include one or more of:        weather widget 149-1, stocks widget 149-2, calculator widget        149-3, alarm clock widget 149-4, dictionary widget 149-5, and        other widgets obtained by the user, as well as user-created        widgets 149-6;    -   digital personal assistant module 150;    -   vehicle integration module 151;    -   video and music player module 152, which is, optionally, made up        of a video player module and a music player module;    -   notes module 153;    -   map module 154; and/or    -   online video module 155.

Examples of other applications 136 that are, optionally, stored inmemory 102 include other word processing applications, other imageediting applications, drawing applications, presentation applications,JAVA-enabled applications, encryption, digital rights management, voicerecognition, and voice replication.

In conjunction with touch screen 112, display controller 156, contactmodule 130, graphics module 132, and text input module 134, contactsmodule 137 are, optionally, used to manage an address book or contactlist (e.g., stored in application internal state 192 of contacts module137 in memory 102 or memory 370), including: adding name(s) to theaddress book; deleting name(s) from the address book; associatingtelephone number(s), e-mail address(es), physical address(es) or otherinformation with a name; associating an image with a name; categorizingand sorting names; providing telephone numbers or e-mail addresses toinitiate and/or facilitate communications by telephone 138, videoconference 139, e-mail 140, or IM 141; and so forth.

In conjunction with RF circuitry 108, audio circuitry 110, speaker 111,microphone 113, touch screen 112, display controller 156, contact module130, graphics module 132, and text input module 134, telephone module138 are, optionally, used to enter a sequence of characterscorresponding to a telephone number, access one or more telephonenumbers in address book 137, modify a telephone number that has beenentered, dial a respective telephone number, conduct a conversation anddisconnect or hang up when the conversation is completed. As notedabove, the wireless communication optionally uses any of a plurality ofcommunications standards, protocols and technologies.

In conjunction with RF circuitry 108, audio circuitry 110, speaker 111,microphone 113, touch screen 112, display controller 156, optical sensor164, optical sensor controller 158, contact module 130, graphics module132, text input module 134, contact list 137, and telephone module 138,videoconferencing module 139 includes executable instructions toinitiate, conduct, and terminate a video conference between a user andone or more other participants in accordance with user instructions.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact module 130, graphics module 132, and text inputmodule 134, e-mail client module 140 includes executable instructions tocreate, send, receive, and manage e-mail in response to userinstructions. In conjunction with image management module 144, e-mailclient module 140 makes it very easy to create and send e-mails withstill or video images taken with camera module 143.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact module 130, graphics module 132, and text inputmodule 134, the instant messaging module 141 includes executableinstructions to enter a sequence of characters corresponding to aninstant message, to modify previously entered characters, to transmit arespective instant message (e.g., using a Short Message Service (SMS) orMultimedia Message Service (MMS) protocol for telephony-based instantmessages or using XMPP, SIMPLE, or IMPS for Internet-based instantmessages), to receive instant messages and to view received instantmessages. In some embodiments, transmitted and/or received instantmessages optionally include graphics, photos, audio files, video filesand/or other attachments as are supported in a MMS and/or an EnhancedMessaging Service (EMS). As used herein, “instant messaging” refers toboth telephony-based messages (e.g., messages sent using SMS or MMS) andInternet-based messages (e.g., messages sent using XMPP, SIMPLE, orIMPS).

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact module 130, graphics module 132, text inputmodule 134, GPS module 135, map module 154, and music player module 146,workout support module 142 includes executable instructions to createworkouts (e.g., with time, distance, and/or calorie burning goals);communicate with workout sensors (sports devices); receive workoutsensor data; calibrate sensors used to monitor a workout; select andplay music for a workout; and display, store and transmit workout data.

In conjunction with touch screen 112, display controller 156, opticalsensor(s) 164, optical sensor controller 158, contact module 130,graphics module 132, and image management module 144, camera module 143includes executable instructions to capture still images or video(including a video stream) and store them into memory 102, modifycharacteristics of a still image or video, or delete a still image orvideo from memory 102.

In conjunction with touch screen 112, display controller 156, contactmodule 130, graphics module 132, text input module 134, and cameramodule 143, image management module 144 includes executable instructionsto arrange, modify (e.g., edit), or otherwise manipulate, label, delete,present (e.g., in a digital slide show or album), and store still and/orvideo images.

In conjunction with RF circuitry 108, touch screen 112, display systemcontroller 156, contact module 130, graphics module 132, and text inputmodule 134, browser module 147 includes executable instructions tobrowse the Internet in accordance with user instructions, includingsearching, linking to, receiving, and displaying web pages or portionsthereof, as well as attachments and other files linked to web pages.

In conjunction with RF circuitry 108, touch screen 112, display systemcontroller 156, contact module 130, graphics module 132, text inputmodule 134, e-mail client module 140, and browser module 147, calendarmodule 148 includes executable instructions to create, display, modify,and store calendars and data associated with calendars (e.g., calendarentries, to do lists, etc.) in accordance with user instructions.

In conjunction with RF circuitry 108, touch screen 112, display systemcontroller 156, contact module 130, graphics module 132, text inputmodule 134, and browser module 147, widget modules 149 aremini-applications that are, optionally, downloaded and used by a user(e.g., weather widget 149-1, stocks widget 149-2, calculator widget149-3, alarm clock widget 149-4, and dictionary widget 149-5) or createdby the user (e.g., user-created widget 149-6). In some embodiments, awidget includes an HTML (Hypertext Markup Language) file, a CSS(Cascading Style Sheets) file, and a JavaScript file. In someembodiments, a widget includes an XML (Extensible Markup Language) fileand a JavaScript file (e.g., Yahoo!® Widgets).

In conjunction with RF circuitry 108, audio circuitry 110, speaker 111,microphone 113, touch screen 112, display controller 156, contact module130, graphics module 132, and text input module 134, digital personalassistant module 150 records voice commands and sends informationrepresentative of the recorded voice commands to a server such as server510 in FIG. 5A for analysis, and responds to the voice commands based ona response from the server.

Vehicle integration module 151 includes executable instructions for oneor more intermediation processes that control a vehicle informationdisplay system in a vehicle (e.g., a car, a truck, a van, etc.) thatprovides a user interface on a respective display of the vehicleinformation display system (e.g., display 546 of peripheral display unit540 in FIG. 5A), such as for a mapping application or a musicapplication. The vehicle integration application converts informationfrom third-party applications into content for display by the vehicleintegration application on the respective display of the vehicleinformation display system.

In conjunction with touch screen 112, display system controller 156,contact module 130, graphics module 132, audio circuitry 110, speaker111, RF circuitry 108, and browser module 147, video and music playermodule 152 includes executable instructions that allow the user todownload and play back recorded music and other sound files stored inone or more file formats, such as MP3 or AAC files, and executableinstructions to display, present or otherwise play back videos (e.g., ontouch screen 112 or on an external, connected display via external port124). In some embodiments, device 100 optionally includes thefunctionality of an MP3 player, such as an iPod (trademark of AppleInc.).

In conjunction with touch screen 112, display controller 156, contactmodule 130, graphics module 132, and text input module 134, notes module153 includes executable instructions to create and manage notes, to dolists, and the like in accordance with user instructions.

In conjunction with RF circuitry 108, touch screen 112, display systemcontroller 156, contact module 130, graphics module 132, text inputmodule 134, GPS module 135, and browser module 147, map module 154 are,optionally, used to receive, display, modify, and store maps and dataassociated with maps (e.g., driving directions; data on stores and otherpoints of interest at or near a particular location; and otherlocation-based data) in accordance with user instructions.

In conjunction with touch screen 112, display system controller 156,contact module 130, graphics module 132, audio circuitry 110, speaker111, RF circuitry 108, text input module 134, e-mail client module 140,and browser module 147, online video module 155 includes instructionsthat allow the user to access, browse, receive (e.g., by streamingand/or download), play back (e.g., on the touch screen or on anexternal, connected display via external port 124), send an e-mail witha link to a particular online video, and otherwise manage online videosin one or more file formats, such as H.264. In some embodiments, instantmessaging module 141, rather than e-mail client module 140, is used tosend a link to a particular online video.

Each of the above identified modules and applications correspond to aset of executable instructions for performing one or more functionsdescribed above and the methods described in this application (e.g., thecomputer-implemented methods and other information processing methodsdescribed herein). These modules (e.g., sets of instructions) need notbe implemented as separate software programs, procedures or modules, andthus various subsets of these modules are, optionally, combined orotherwise re-arranged in various embodiments. In some embodiments,memory 102 optionally stores a subset of the modules and data structuresidentified above. Furthermore, memory 102 optionally stores additionalmodules and data structures not described above.

In some embodiments, device 100 is a device where operation of apredefined set of functions on the device is performed exclusivelythrough a touch screen and/or a touchpad. By using a touch screen and/ora touchpad as the primary input control device for operation of device100, the number of physical input control devices (such as push buttons,dials, and the like) on device 100 is, optionally, reduced.

The predefined set of functions that are performed exclusively through atouch screen and/or a touchpad optionally include navigation betweenuser interfaces. In some embodiments, the touchpad, when touched by theuser, navigates device 100 to a main, home, or root menu from any userinterface that is displayed on device 100. In such embodiments, a “menubutton” is implemented using a touchpad. In some other embodiments, themenu button is a physical push button or other physical input controldevice instead of a touchpad.

FIG. 1B is a block diagram illustrating exemplary components for eventhandling in accordance with some embodiments. In some embodiments,memory 102 (in FIG. 1A) or 370 (FIG. 3) includes event sorter 170 (e.g.,in operating system 126) and a respective application 136-1 (e.g., anyof the aforementioned applications 137-13, 155, 380-390).

Event sorter 170 receives event information and determines theapplication 136-1 and application view 191 of application 136-1 to whichto deliver the event information. Event sorter 170 includes eventmonitor 171 and event dispatcher module 174. In some embodiments,application 136-1 includes application internal state 192, whichindicates the current application view(s) displayed on touch sensitivedisplay 112 when the application is active or executing. In someembodiments, device/global internal state 157 is used by event sorter170 to determine which application(s) is (are) currently active, andapplication internal state 192 is used by event sorter 170 to determineapplication views 191 to which to deliver event information.

In some embodiments, application internal state 192 includes additionalinformation, such as one or more of: resume information to be used whenapplication 136-1 resumes execution, user interface state informationthat indicates information being displayed or that is ready for displayby application 136-1, a state queue for enabling the user to go back toa prior state or view of application 136-1, and a redo/undo queue ofprevious actions taken by the user.

Event monitor 171 receives event information from peripherals interface118. Event information includes information about a sub-event (e.g., auser touch on touch-sensitive display 112, as part of a multi-touchgesture). Peripherals interface 118 transmits information it receivesfrom I/O subsystem 106 or a sensor, such as proximity sensor 166,accelerometer(s) 168, and/or microphone 113 (through audio circuitry110). Information that peripherals interface 118 receives from I/Osubsystem 106 includes information from touch-sensitive display 112 or atouch-sensitive surface.

In some embodiments, event monitor 171 sends requests to the peripheralsinterface 118 at predetermined intervals. In response, peripheralsinterface 118 transmits event information. In other embodiments,peripheral interface 118 transmits event information only when there isa significant event (e.g., receiving an input above a predeterminednoise threshold and/or for more than a predetermined duration).

In some embodiments, event sorter 170 also includes a hit viewdetermination module 172 and/or an active event recognizer determinationmodule 173.

Hit view determination module 172 provides software procedures fordetermining where a sub-event has taken place within one or more views,when touch sensitive display 112 displays more than one view. Views aremade up of controls and other elements that a user can see on thedisplay.

Another aspect of the user interface associated with an application is aset of views, sometimes herein called application views or userinterface windows, in which information is displayed and touch-basedgestures occur. The application views (of a respective application) inwhich a touch is detected optionally correspond to programmatic levelswithin a programmatic or view hierarchy of the application. For example,the lowest level view in which a touch is detected is, optionally,called the hit view, and the set of events that are recognized as properinputs are, optionally, determined based, at least in part, on the hitview of the initial touch that begins a touch-based gesture.

Hit view determination module 172 receives information related tosub-events of a touch-based gesture. When an application has multipleviews organized in a hierarchy, hit view determination module 172identifies a hit view as the lowest view in the hierarchy which shouldhandle the sub-event. In most circumstances, the hit view is the lowestlevel view in which an initiating sub-event occurs (e.g., the firstsub-event in the sequence of sub-events that form an event or potentialevent). Once the hit view is identified by the hit view determinationmodule, the hit view typically receives all sub-events related to thesame touch or input source for which it was identified as the hit view.

Active event recognizer determination module 173 determines which viewor views within a view hierarchy should receive a particular sequence ofsub-events. In some embodiments, active event recognizer determinationmodule 173 determines that only the hit view should receive a particularsequence of sub-events. In other embodiments, active event recognizerdetermination module 173 determines that all views that include thephysic allocation of a sub-event are actively involved views, andtherefore determines that all actively involved views should receive aparticular sequence of sub-events. In other embodiments, even if touchsub-events were entirely confined to the area associated with oneparticular view, views higher in the hierarchy would still remain asactively involved views.

Event dispatcher module 174 dispatches the event information to an eventrecognizer (e.g., event recognizer 180). In embodiments including activeevent recognizer determination module 173, event dispatcher module 174delivers the event information to an event recognizer determined byactive event recognizer determination module 173. In some embodiments,event dispatcher module 174 stores in an event queue the eventinformation, which is retrieved by a respective event receiver module182.

In some embodiments, operating system 126 includes event sorter 170.Alternatively, application 136-1 includes event sorter 170. In yet otherembodiments, event sorter 170 is a stand-alone module, or a part ofanother module stored in memory 102, such as contact/motion module 130.

In some embodiments, application 136-1 includes a plurality of eventhandlers 190 and one or more application views 191, each of whichincludes instructions for handling touch events that occur within arespective view of the application's user interface. Each applicationview 191 of the application 136-1 includes one or more event recognizers180. Typically, a respective application view 191 includes a pluralityof event recognizers 180. In other embodiments, one or more of eventrecognizers 180 are part of a separate module, such as a user interfacekit (not shown) or a higher level object from which application 136-1inherits methods and other properties. In some embodiments, a respectiveevent handler 190 includes one or more of: data updater 176, objectupdater 177, GUI updater 178, and/or event data 179 received from eventsorter 170. Event handler 190 optionally utilizes or calls data updater176, object updater 177 or GUI updater 178 to update the applicationinternal state 192. Alternatively, one or more of the application views191 includes one or more respective event handlers 190. Also, in someembodiments, one or more of data updater 176, object updater 177, andGUI updater 178 are included in a respective application view 191.

A respective event recognizer 180 receives event information (e.g.,event data 179) from event sorter 170, and identifies an event from theevent information. Event recognizer 180 includes event receiver 182 andevent comparator 184. In some embodiments, event recognizer 180 alsoincludes at least a subset of: metadata 183, and event deliveryinstructions 188 (which optionally include sub-event deliveryinstructions).

Event receiver 182 receives event information from event sorter 170. Theevent information includes information about a sub-event, for example, atouch or a touch movement. Depending on the sub-event, the eventinformation also includes additional information, such as location ofthe sub-event. When the sub-event concerns motion of a touch, the eventinformation optionally also includes speed and direction of thesub-event. In some embodiments, events include rotation of the devicefrom one orientation to another (e.g., from a portrait orientation to alandscape orientation, or vice versa), and the event informationincludes corresponding information about the current orientation (alsocalled device attitude) of the device.

Event comparator 184 compares the event information to predefined eventor sub-event definitions and, based on the comparison, determines anevent or sub-event, or determines or updates the state of an event orsub-event. In some embodiments, event comparator 184 includes eventdefinitions 186. Event definitions 186 contain definitions of events(e.g., predefined sequences of sub-events), for example, event 1(187-1), event 2 (187-2), and others. In some embodiments, sub-events inan event 187 include, for example, touch begin, touch end, touchmovement, touch cancellation, and multiple touching. In one example, thedefinition for event 1 (187-1) is a double tap on a displayed object.The double tap, for example, comprises a first touch (touch begin) onthe displayed object for a predetermined phase, a first lift-off (touchend) for a predetermined phase, a second touch (touch begin) on thedisplayed object for a predetermined phase, and a second lift-off (touchend) for a predetermined phase. In another example, the definition forevent 2 (187-2) is a dragging on a displayed object. The dragging, forexample, comprises a touch (or contact) on the displayed object for apredetermined phase, a movement of the touch across touch-sensitivedisplay 112, and lift-off of the touch (touch end). In some embodiments,the event also includes information for one or more associated eventhandlers 190.

In some embodiments, event definition 187 includes a definition of anevent for a respective user-interface object. In some embodiments, eventcomparator 184 performs a hit test to determine which user-interfaceobject is associated with a sub-event. For example, in an applicationview in which three user-interface objects are displayed ontouch-sensitive display 112, when a touch is detected on touch-sensitivedisplay 112, event comparator 184 performs a hit test to determine whichof the three user-interface objects is associated with the touch(sub-event). If each displayed object is associated with a respectiveevent handler 190, the event comparator uses the result of the hit testto determine which event handler 190 should be activated. For example,event comparator 184 selects an event handler associated with thesub-event and the object triggering the hit test.

In some embodiments, the definition for a respective event 187 alsoincludes delayed actions that delay delivery of the event informationuntil after it has been determined whether the sequence of sub-eventsdoes or does not correspond to the event recognizer's event type.

When a respective event recognizer 180 determines that the series ofsub-events do not match any of the events in event definitions 186, therespective event recognizer 180 enters an event impossible, eventfailed, or event ended state, after which it disregards subsequentsub-events of the touch-based gesture. In this situation, other eventrecognizers, if any, that remain active for the hit view continue totrack and process sub-events of an ongoing touch-based gesture.

In some embodiments, a respective event recognizer 180 includes metadata183 with configurable properties, flags, and/or lists that indicate howthe event delivery system should perform sub-event delivery to activelyinvolved event recognizers. In some embodiments, metadata 183 includesconfigurable properties, flags, and/or lists that indicate how eventrecognizers interact, or are enabled to interact, with one another. Insome embodiments, metadata 183 includes configurable properties, flags,and/or lists that indicate whether sub-events are delivered to varyinglevels in the view or programmatic hierarchy.

In some embodiments, a respective event recognizer 180 activates eventhandler 190 associated with an event when one or more particularsub-events of an event are recognized. In some embodiments, a respectiveevent recognizer 180 delivers event information associated with theevent to event handler 190. Activating an event handler 190 is distinctfrom sending (and deferred sending) sub-events to a respective hit view.In some embodiments, event recognizer 180 throws a flag associated withthe recognized event, and event handler 190 associated with the flagcatches the flag and performs a predefined process.

In some embodiments, event delivery instructions 188 include sub-eventdelivery instructions that deliver event information about a sub-eventwithout activating an event handler. Instead, the sub-event deliveryinstructions deliver event information to event handlers associated withthe series of sub-events or to actively involved views. Event handlersassociated with the series of sub-events or with actively involved viewsreceive the event information and perform a predetermined process.

In some embodiments, data updater 176 creates and updates data used inapplication 136-1. For example, data updater 176 updates the telephonenumber used in contacts module 137, or stores a video file used in videoplayer module 145. In some embodiments, object updater 177 creates andupdates objects used in application 136-1. For example, object updater177 creates a new user-interface object or updates the position of auser-interface object. GUI updater 178 updates the GUI. For example, GUIupdater 178 prepares display information and sends it to graphics module132 for display on a touch-sensitive display.

In some embodiments, event handler(s) 190 includes or has access to dataupdater 176, object updater 177, and GUI updater 178. In someembodiments, data updater 176, object updater 177, and GUI updater 178are included in a single module of a respective application 136-1 orapplication view 191. In other embodiments, they are included in two ormore software modules.

It shall be understood that the foregoing discussion regarding eventhandling of user touches on touch-sensitive displays also applies toother forms of user inputs to operate multifunction devices 100 withinput-devices, not all of which are initiated on touch screens. Forexample, mouse movement and mouse button presses, optionally coordinatedwith single or multiple keyboard presses or holds; contact movementssuch as taps, drags, scrolls, etc., on touch-pads; pen stylus inputs;movement of the device; oral instructions; detected eye movements;biometric inputs; and/or any combination thereof are optionally utilizedas inputs corresponding to sub-events which define an event to berecognized.

FIG. 2 illustrates a portable multifunction device 100 having a touchscreen 112 in accordance with some embodiments. The touch screenoptionally displays one or more graphics within user interface (UI) 200.In this embodiment, as well as others described below, a user is enabledto select one or more of the graphics by making a gesture on thegraphics, for example, with one or more fingers 202 (not drawn to scalein the figure) or one or more styluses 203 (not drawn to scale in thefigure). In some embodiments, selection of one or more graphics occurswhen the user breaks contact with the one or more graphics. In someembodiments, the gesture optionally includes one or more taps, one ormore swipes (from left to right, right to left, upward and/or downward)and/or a rolling of a finger (from right to left, left to right, upwardand/or downward) that has made contact with device 100. In someimplementations or circumstances, inadvertent contact with a graphicdoes not select the graphic. For example, a swipe gesture that sweepsover an application icon optionally does not select the correspondingapplication when the gesture corresponding to selection is a tap.

Device 100 optionally also includes one or more physical buttons, suchas “home” or menu button 204. As described previously, menu button 204is, optionally, used to navigate to any application 136 in a set ofapplications that are, optionally executed on device 100. Alternatively,in some embodiments, the menu button is implemented as a soft key in aGUI displayed on touch screen 112.

In one embodiment, device 100 includes touch screen 112, menu button204, push button 206 for powering the device on/off and locking thedevice, volume adjustment button(s) 208, Subscriber Identity Module(SIM) card slot 210, head set jack 212, and docking/charging externalport 124. Push button 206 is, optionally, used to turn the power on/offon the device by depressing the button and holding the button in thedepressed state for a predefined time interval; to lock the device bydepressing the button and releasing the button before the predefinedtime interval has elapsed; and/or to unlock the device or initiate anunlock process. In an alternative embodiment, device 100 also acceptsverbal input for activation or deactivation of some functions throughmicrophone 113. Device 100 also, optionally, includes one or morecontact intensity sensors 165 for detecting intensity of contacts ontouch screen 112 and/or one or more tactile output generators 167 forgenerating tactile outputs for a user of device 100.

FIG. 3 is a block diagram of an exemplary multifunction device with adisplay and a touch-sensitive surface in accordance with someembodiments. Device 300 need not be portable. In some embodiments,device 300 is a laptop computer, a desktop computer, a tablet computer,a multimedia player device, a navigation device, an educational device(such as a child's learning toy), a gaming system, or a control device(e.g., a home or industrial controller). Device 300 typically includesone or more processing units (CPU's) 310, one or more network or othercommunications interfaces 360, memory 370, and one or more communicationbuses 320 for interconnecting these components. Communication buses 320optionally include circuitry (sometimes called a chipset) thatinterconnects and controls communications between system components.Device 300 includes input/output (I/O) interface 330 comprising display340, which is typically a touch screen display (e.g., touch screendisplay 112). I/O interface 330 also optionally includes a keyboardand/or mouse (or other pointing device) 350 and touchpad 355, tactileoutput generator 357 for generating tactile outputs on device 300 (e.g.,similar to tactile output generator(s) 167 described above withreference to FIG. 1A), sensors 359 (e.g., optical, acceleration,proximity, touch-sensitive, and/or contact intensity sensors similar tocontact intensity sensor(s) 165 described above with reference to FIG.1A). Memory 370 includes high-speed random access memory, such as DRAM,SRAM, DDR RAM or other random access solid state memory devices; andoptionally includes non-volatile memory, such as one or more magneticdisk storage devices, optical disk storage devices, flash memorydevices, or other non-volatile solid state storage devices. Memory 370optionally includes one or more storage devices remotely located fromCPU(s) 310. In some embodiments, memory 370 stores programs, modules,and data structures analogous to the programs, modules, and datastructures stored in memory 102 of portable multifunction device 100(FIG. 1A), or a subset thereof. Furthermore, memory 370 optionallystores additional programs, modules, and data structures not present inmemory 102 of portable multifunction device 100. For example, memory 370of device 300 optionally stores drawing module 380, presentation module382, word processing module 384, website creation module 386, diskauthoring module 388, and/or spreadsheet module 390, while memory 102 ofportable multifunction device 100 (FIG. 1A) optionally does not storethese modules.

Each of the above identified elements in FIG. 3 are, optionally, storedin one or more of the previously mentioned memory devices. Each of theabove identified modules corresponds to a set of instructions forperforming a function described above. The above identified modules orprograms (e.g., sets of instructions) need not be implemented asseparate software programs, procedures or modules, and thus varioussubsets of these modules are, optionally, combined or otherwisere-arranged in various embodiments. In some embodiments, memory 370optionally stores a subset of the modules and data structures identifiedabove. Furthermore, memory 370 optionally stores additional modules anddata structures not described above.

Attention is now directed towards embodiments of user interfaces (“UI”)that is, optionally, implemented on portable multifunction device 100.

FIG. 4A illustrates an exemplary user interface for a menu ofapplications on portable multifunction device 100 in accordance withsome embodiments. Similar user interfaces are, optionally, implementedon device 300. In some embodiments, user interface 400 includes thefollowing elements, or a subset or superset thereof:

-   -   Signal strength indicator(s) 402 for wireless communication(s),        such as cellular and Wi-Fi signals;    -   Time 404;    -   Bluetooth indicator 405;    -   Battery status indicator 406;    -   Tray 408 with icons for frequently used applications, such as:        -   Icon 416 for telephone module 138, labeled “Phone,” which            optionally includes an indicator 414 of the number of missed            calls or voicemail messages;        -   Icon 418 for e-mail client module 140, labeled “Mail,” which            optionally includes an indicator 410 of the number of unread            e-mails;        -   Icon 420 for browser module 147, labeled “Browser”; and        -   Icon 422 for video and music player module 152, also            referred to as iPod (trademark of Apple Inc.) module 152,            labeled “iPod.”    -   Icons for other applications, such as:        -   Icon 424 for IM module 141, labeled “Text”;        -   Icon 426 for calendar module 148, labeled “Calendar”;        -   Icon 428 for image management module 144, labeled “Photos”;        -   Icon 430 for camera module 143, labeled “Camera”;        -   Icon 432 for online video module 155, labeled “Online            Video”;        -   Icon 434 for stocks widget 149-2, labeled “Stocks”;        -   Icon 436 for map module 154, labeled “Map”;        -   Icon 438 for weather widget 149-1, labeled “Weather”;        -   Icon 440 for alarm clock widget 149-4, labeled “Clock”;        -   Icon 442 for workout support module 142, labeled “Workout            Support”;        -   Icon 444 for notes module 153, labeled “Notes”; and        -   Icon 446 for a settings application or module, which            provides access to settings for device 100 and its various            applications 136.

It should be noted that the icon labels illustrated in FIG. 4A aremerely exemplary. For example, icon 422 for video and music playermodule 152 are labeled “Music” or “Music Player.” Other labels are,optionally, used for various application icons. In some embodiments, alabel for a respective application icon includes a name of anapplication corresponding to the respective application icon. In someembodiments, a label for a particular application icon is distinct froma name of an application corresponding to the particular applicationicon.

FIG. 4B illustrates an exemplary user interface on a device (e.g.,device 300, FIG. 3) with a touch-sensitive surface 451 (e.g., a tabletor touchpad 355, FIG. 3) that is separate from the display 450 (e.g.,touch screen display 112). Device 300 also, optionally, includes one ormore contact intensity sensors (e.g., one or more of sensors 357) fordetecting intensity of contacts on touch-sensitive surface 451 and/orone or more tactile output generators 359 for generating tactile outputsfor a user of device 300.

Although some of the examples which follow will be given with referenceto inputs on touch screen display 112 (where the touch sensitive surfaceand the display are combined), in some embodiments, the device detectsinputs on a touch-sensitive surface that is separate from the display,as shown in FIG. 4B. In some embodiments the touch sensitive surface(e.g., 451 in FIG. 4B) has a primary axis (e.g., 452 in FIG. 4B) thatcorresponds to a primary axis (e.g., 453 in FIG. 4B) on the display(e.g., 450). In accordance with these embodiments, the device detectscontacts (e.g., 460 and 462 in FIG. 4B) with the touch-sensitive surface451 at locations that correspond to respective locations on the display(e.g., in FIG. 4B, 460 corresponds to 468 and 462 corresponds to 470).In this way, user inputs (e.g., contacts 460 and 462, and movementsthereof) detected by the device on the touch-sensitive surface (e.g.,451 in FIG. 4B) are used by the device to manipulate the user interfaceon the display (e.g., 450 in FIG. 4B) of the multifunction device whenthe touch-sensitive surface is separate from the display. It should beunderstood that similar methods are, optionally, used for other userinterfaces described herein.

Additionally, while the following examples are given primarily withreference to finger inputs (e.g., finger contacts, finger tap gestures,finger swipe gestures), it should be understood that, in someembodiments, one or more of the finger inputs are replaced with inputfrom another input device (e.g., a mouse based input or stylus input).For example, a swipe gesture is, optionally, replaced with a mouse click(e.g., instead of a contact) followed by movement of the cursor alongthe path of the swipe (e.g., instead of movement of the contact). Asanother example, a tap gesture is, optionally, replaced with a mouseclick while the cursor is located over the location of the tap gesture(e.g., instead of detection of the contact followed by ceasing to detectthe contact). Similarly, when multiple user inputs are simultaneouslydetected, it should be understood that multiple computer mice are,optionally, used simultaneously, or a mouse and finger contacts are,optionally, used simultaneously.

As used herein, the term “focus selector” refers to an input elementthat indicates a current part of a user interface with which a user isinteracting. In some implementations that include a cursor or otherlocation marker, the cursor acts as a “focus selector,” so that when aninput (e.g., a press input) is detected on a touch-sensitive surface(e.g., touchpad 355 in FIG. 3 or touch-sensitive surface 451 in FIG. 4B)while the cursor is over a particular user interface element (e.g., abutton, window, slider or other user interface element), the particularuser interface element is adjusted in accordance with the detectedinput. In some implementations that include a touch screen display(e.g., touch-sensitive display system 112 in FIG. 1A or touch screen 112in FIG. 4A) that enables direct interaction with user interface elementson the touch screen display, a detected contact on the touch screen actsas a “focus selector,” so that when an input (e.g., a press input by thecontact) is detected on the touch screen display at a location of aparticular user interface element (e.g., a button, window, slider orother user interface element), the particular user interface element isadjusted in accordance with the detected input. In some implementationsfocus is moved from one region of a user interface to another region ofthe user interface without corresponding movement of a cursor ormovement of a contact on a touch screen display (e.g., by using a tabkey or arrow keys to move focus from one button to another button); inthese implementations, the focus selector moves in accordance withmovement of focus between different regions of the user interface.Without regard to the specific form taken by the focus selector, thefocus selector is generally the user interface element (or contact on atouch screen display) that is controlled by the user so as tocommunicate the user's intended interaction with the user interface(e.g., by indicating, to the device, the element of the user interfacewith which the user is intending to interact). For example, the locationof a focus selector (e.g., a cursor, a contact or a selection box) overa respective button while a press input is detected on thetouch-sensitive surface (e.g., a touchpad or touch screen) will indicatethat the user is intending to activate the respective button (as opposedto other user interface elements shown on a display of the device).

FIG. 5A illustrates a block diagram of an operating environment 500 inaccordance with some embodiments. Operating environment 500 includes aserver 510, one or more communications networks 505, portablemultifunction device 100, and peripheral display unit 540. In someembodiments, peripheral display unit 540 is an entertainment and/ornavigation system that is implemented in a vehicle. In some embodiments,peripheral display unit 540 includes one or more displays. In someembodiments, a vehicle includes a plurality of peripheral display unit540 communicatively coupled to device 100 in operating environment 500each with a respective display. In some embodiments, peripheral displayunit 540 is a peripheral display unit.

Server 510 typically includes one or more processing units (CPUs) 512for executing modules, programs and/or instructions stored in memory 524and thereby performing processing operations, one or more network orother communications interfaces 520, memory 524, and one or morecommunication buses 522 for interconnecting these components.Communication buses 522 optionally include circuitry (sometimes called achipset) that interconnects and controls communications between systemcomponents. Memory 524 includes high-speed random access memory, such asDRAM, SRAM, DDR RAM or other random access solid state memory devices,and can optionally include non-volatile memory, such as one or moremagnetic disk storage devices, optical disk storage devices, flashmemory devices, or other non-volatile solid state storage devices.Memory 524 optionally includes one or more storage devices remotelylocated from the CPU(s) 512. Memory 524, or alternately the non-volatilememory device(s) within memory 524, comprises a non-transitorycomputer-readable storage medium. In some embodiments, memory 524, orthe computer-readable storage medium of memory 524 stores the followingprograms, modules, and data structures, or a subset thereof:

-   -   an operating system 526 that includes procedures for handling        various basic system services and for performing hardware        dependent tasks; and    -   a network communication module 528 that is used for connecting        (wired or wireless) server 510 to other computing devices via        the one or more communication network interfaces 520 and one or        more communication networks 505, such as the Internet, other        wide area networks, local area networks, metropolitan area        networks, and so on.

Portable multifunction device 100 (sometimes herein also called “device100”) typically includes the components described with reference toFIGS. 1A-1B and/or 3.

Peripheral display unit 540 (sometimes herein also called a“peripheral”) typically includes one or more processing units (CPUs) 542for executing modules, programs and/or instructions stored in memory 554and thereby performing processing operations, one or more network orother communications interfaces 550, memory 554, and one or morecommunication buses 552 for interconnecting these components.

Communication interface 550 can optionally include a plurality ofdiscrete communication interfaces capable of communicating independentlyand simultaneously. For example, communication interface 550 canoptionally include one or more wired communication interfaces such as aUSB data port. Communication interface 550 can optionally furtherinclude one or more wireless communication interfaces, such as a Wi-Ficommunication interface and/or a Bluetooth communication interface.Communication interface 550 can optionally include additionalcommunication interfaces.

Peripheral display unit 540, optionally, includes a user interface 544comprising one or more display devices 546 and a plurality of controls548 (e.g., jog dials, knobs, buttons, switches, a touch-sensitivesurface such as a touch screen display, or other input sources). In someembodiments, the one or more displays 546 include a primary display546-1 (e.g., a dashboard or vehicle navigation display) and an auxiliarydisplay 546-2 (e.g., a rear-seat or entertainment display). In someembodiments, a respective display of the one or more displays 546 is atouch screen display that is capable of receiving user touch inputs(e.g., detecting finger contacts and gestures that correspond to thedetection and movement of finger contacts). In some embodiments, arespective display of the one or more displays 546 is associated withone or more controls of the plurality of controls 548 (e.g., jog dials,knobs, buttons, switches, a touch-sensitive surface such as a touchscreen display, or other input sources.). Communication buses 552optionally include circuitry (sometimes called a chipset) thatinterconnects and controls communications between system components.Memory 554 includes high-speed random access memory, such as DRAM, SRAM,DDR RAM or other random access solid state memory devices, and,optionally, includes non-volatile memory, such as one or more magneticdisk storage devices, optical disk storage devices, flash memorydevices, or other non-volatile solid state storage devices. Memory 554optionally includes one or more storage devices remotely located fromthe CPU(s) 542. Memory 552, or alternately the non-volatile memorydevice(s) within memory 552, comprises a non-transitorycomputer-readable storage medium. In some embodiments, memory 552, orthe computer-readable storage medium of memory 552 stores the followingprograms, modules, and data structures, or a subset thereof.

-   -   an operating system 556 that includes procedures for handling        various basic system services and for performing hardware        dependent tasks; and    -   a network communication module 558 that is used for connecting        (wired or wireless) server 540 to other computing devices via        the one or more communication network interfaces 550 and one or        more communication networks 505, such as the Internet, other        wide area networks, local area networks, metropolitan area        networks, and so on.

In some embodiments, device 100 drives the one or more displays 546 ofperipheral display unit 540. For example, device 100 sends a videosignal to peripheral display unit 540, and CPU 542 of peripheral displayunit 540 renders the video signal on the one or more displays 546. Insome embodiments, device 100 sends a video signal directly to the one ormore displays 546 and CPU 542 is not used to render the video signal(e.g., device 100 uses display 546 as an auxiliary display). In someembodiments, the user interface displayed on touch screen 112 of device100 is synchronized with the user interface displayed on the one or moredisplays 546 of peripheral display unit 540, and, in some otherembodiments, the user interface displayed on touch screen 112 of device100 is not continuously synchronized with the user interface displayedon the one or more displays 546 of peripheral display unit 540 and attimes (e.g., while a user of device 100 is viewing information aboutpossible driving destinations but has not yet selected a drivingdestination) touchscreen 112 of device 100 displays differentinformation from that displayed on display 546 of peripheral displayunit 540 (e.g., touch screen 112 and display 546 are intermittentlysynchronized with periods in between the intermittent synchronizationevents where they are not synchronized).

In some embodiments, in response to detecting a user input (e.g., a usertouch input associated with a respective display of the one or moredisplays 546 or a user input associated with a respective control of theplurality of controls 548), peripheral display unit 540 (or therespective display of the one or more displays 546, or the respectivecontrol of the plurality of controls 548) sends input information (e.g.,an identifier for the input source and an input description describingthe user input) corresponding to the user input to device 100. In turn,device 100 updates the user interface displayed on the respectivedisplay of the one or more displays 546 and/or touch screen 112 ofdevice 100 in accordance with the received input information and/or thedisplay state of the user interface displayed on the respective displayof the one or more displays 546 at or before the user input.

FIG. 5B is a flow diagram illustrating a process of selecting anaffected display and sending update information to the affected display.The affected display (e.g., primary display 546-1) displays (560) a userinterface. A respective input source that is a control of peripheraldisplay unit 540 detects (561) a user input. In some embodiments, therespective input source is one of controls 548 and the user input is aninteraction with one of controls 548. For example, controls 548 includebuttons, switches, dials, knobs, other mechanical affordances,touch-sensitive surfaces, or other input sources. For example, the userinput is rotation of a knob or dial, depression of an affordance orknob, a touch input detected on a touch-sensitive surface or touchscreen, or other user interaction with controls 548.

In response to detecting the user input, the respective input sourcesends (562) input information (e.g., {Source ID; Input Description}) todevice 100. In some embodiments, the input information includes a uniqueidentifier for the input source and input description information thatdescribes the user input. For example, the input description informationis raw input data such as the magnitude and direction of rotation of ajog dial, contact locations and movement amounts/directions detected ona touch-sensitive surface and/or gesture data describing a type ofgesture that was performed on the touch-sensitive surface.

Device 100 receives (564) the input information from the respectiveinput source. Device 100 selects an affected display by correlating theunique identifier (e.g., source ID) included in the input informationwith a display identifier tag (e.g., a display ID) based at least inpart on input-source mapping 566. In some embodiments, the input-sourcemapping 566 is stored in memory of device 100. In some embodiments,multiple inputs sources are mapped to a same display. However, in someembodiments, each input source is mapped no more than a single display.In some embodiments, input-source mapping 566 is updated by device 100in response to detecting input-source-mapping update events (e.g., avehicle associated with the first display starting to back up and takingcontrol of the first display, or an input associating a jog dial withthe second display instead of the first display) and/or in accordancewith a predetermined schedule.

After selecting the affected display, device 100 determines a respectivestate of the user interface displayed on the affected display bycorrelating the display ID (determined based on input-source mapping566) for the affected display with display state information (e.g., whatkind of user interface is displayed in the display, what user interfaceelements are displayed, and/or which controls are associated with whichfunctions such as volume control or scrolling) based at least in part onuser interface state table 568. In some embodiments, the user interfacestate table 568 is stored in memory of device 100. After determining therespective state of the user interface displayed on the affecteddisplay, device 100 generates (570) an updated user interface for theaffected display in accordance with the respective state of the userinterface displayed on the affected display and the input descriptioninformation, and device 100 sends the updated user interface (orinformation for generating an updated user interface) to the affecteddisplay. In some embodiments, device 100 also updates user interfacestate table 568 so that the state information associated with thedisplay ID corresponding to the affected display reflects the updateduser interface (e.g., for use in responding to subsequent inputinformation received from the input source).

In response to receiving the updated user interface (e.g., updateinformation) from device 100, the affected display updates (574), theuser interface displayed on the affected displayed so as to display theupdated user interface (e.g., by replacing an image of a user interfacepreviously provided by device 100 with an updated image of the userinterface provided by device 100 in the update information).

User Interfaces and Associated Processes

Attention is now directed toward embodiments of user interfaces (UI) andassociated processes that can optionally be implemented on an electronicdevice, such as device 300 or portable multifunction device 100.

FIGS. 6A-6W illustrate exemplary user interfaces. FIGS. 7A-7H, 8A-8D,and 9A-9E are flow diagrams illustrating exemplary methods. The userinterfaces in FIGS. 6A-6W are used to illustrate the processes in FIGS.7A-7H, 8A-8D, and 9A-9E.

Attention is now directed to techniques for establishing a relationshipbetween an electronic device and a peripheral display unit. In someembodiments, these techniques may be useful in operating environments inwhich it is convenient for an electronic device, such as a smart phone,to share content with a peripheral display unit, such as anentertainment/navigation/media system in a vehicle (e.g., an accessorysuch as a car stereo head unit). The relationships established, in someembodiments, include an authorization for the device and the peripheraldisplay unit to establish a data connection with one another.

In some embodiments, a data connection between a device and a peripheraldisplay unit is established. The data connection is optionally called a“connection.” When the data connection is over a wireless dataconnection, it is optionally called a “wireless connection.” When thedata connection is over a wired data connection, it is optionally calleda “wired connection.”

In some embodiments, a relationship is created between a device and aperipheral display unit in which information is exchanged andauthorization to establish a connection is recorded/stored. Therelationship that includes authorization to establish the connection isoptionally called a “relationship.” If the relationship authorizeswireless connection, it is optionally called a “wireless relationship.”If the relationship authorizes wired connection, it is optionally calleda “wired relationship.”

In some embodiments, a setup process to create/establish a relationship(a wired relationship or a wireless relationship) is undertaken. Thesetup process can optionally be executed via a wired or wirelessconnection. Regardless of whether the setup process is undertaken via awired or wireless connection, the relationship established canoptionally itself be either a wired relationship or a wirelessrelationship. A setup process executed over a wired connection isoptionally called “wired setup,” while a setup process executed over awireless connection is optionally called “wireless setup.”

The versatility of various connection techniques, variousrelationships/authorizations, and various setup techniques improvesefficiency and makes the experience more intuitive, more convenient, andquicker for a user.

FIG. 6A depicts an exemplary device 600 and an exemplary peripheraldisplay unit 610, which together can optionally embody the techniquesdescribed herein. In some embodiments, device 600 is device 100 or 300(FIGS. 1A, 3, and 5A). In some embodiments, peripheral display unit 610is peripheral display unit 540 (FIG. 5A). In some embodiments, bothdevice 600 and peripheral display unit 610 exist in operatingenvironment 500 (FIG. 5A).

Device 600 has display 608, which is touch screen 112 (FIG. 1A) ordisplay 340 (FIG. 3A) in some embodiments. Device 600 also comprisesthree communication interfaces: first wireless data interface 602,second wireless data interface 604, and wired data interface 606. Insome embodiments, first wireless data interface 602 is a Wi-Fi datainterface capable of sending and receiving information over the Wi-Ficommunication standard. In some embodiments, second wireless datainterface 604 is a Bluetooth communication interface, capable of sendingand receiving information over the Bluetooth communication standard. Insome embodiments, wired communication interface 606 is a computer busand power connector interface capable of sending and receivinginformation on a wire and capable of sending and/or receiving a batterycharge through the same wire. In some embodiments, the wiredcommunication interface 606 is capable of interfacing with USBconnections.

Peripheral display unit 610 has display 618, which is display 546 (FIG.5A) in some embodiments. Peripheral display unit 610 also compriseshardware button 619, which can optionally be a physical button providedon the housing of peripheral display unit 610 or on some physicalelement connected thereto (as will be discussed later). Peripheraldisplay unit 610 also comprises three communication interfaces: firstwireless data interface 612, second wireless data interface 614, andwired data interface 616. In some embodiments, first wireless datainterface 612 is a Wi-Fi data interface capable of sending and receivinginformation over the Wi-Fi communication standard. In some embodiments,second wireless data interface 614 is a Bluetooth communicationinterface, capable of sending and receiving information over theBluetooth communication standard. In some embodiments, wiredcommunication interface 616 is a computer bus and power connectorinterface capable of sending and receiving information on a wire andcapable of sending and/or receiving a battery charge through the samewire. In some embodiments, the wired communication interface 606 iscapable of interfacing with USB connections. In some embodiments,interfaces 612, 614, and 616 are part of communication interface 550.

All three communication interfaces of device 600 can optionally be usedto communicate with peripheral display unit 610. Namely, first wirelessinterfaces 602 and 612 can optionally communicate with one another,second wireless interfaces 604 and 614 can optionally communicate withone another, and wired interfaces 606 and 616 can optionally communicatewith one another. Each of the three corresponding pairs, whencommunicating between one another, may be said to have formed a “dataconnection,” such as a first wireless data connection or a Wi-Fi dataconnection. These data connections can optionally be established orterminated independently of one another, simultaneously with oneanother, and/or in cooperation with one another.

In some embodiments, peripheral display unit 610 is an accessory, suchas a dash mounted head unit of an entertainment/navigation/media systemprovided in a vehicle. In some such embodiments, peripheral display unit610 can optionally also include, or be operatively coupled with,speakers for audio output and a microphone for audio input. In someembodiments, hardware button 619 can optionally be situated on thesteering wheel of the vehicle for convenient access by a user's thumb.

Below, various techniques for establishing a relationship and operatinga connection between a device and a peripheral display unit aredescribed. The device can optionally be device 600 and the peripheraldisplay unit can optionally be peripheral display unit 610.Specifically, the relationship established can optionally facilitatedata communication over the first wireless data connection, such as aconnection corresponding to first wireless data interfaces 602 and 612.The first wireless data connection can optionally be used to passcontent between device 600 and peripheral display unit 610, includingaudio data, video data, phone call data, navigation data, web-browsingdata, media data, user interface data, user input data, and more. Thisconnection is optionally called a “wireless connection” or a“connection.” When the first data connection is a Wi-Fi connection, itmay be preferable as compared to Bluetooth data connections, in that itis more secure, has a longer range, and has a higher bandwidth. However,a Wi-Fi data connection may not be capable of reestablishing aconnection as quickly as a Bluetooth data connection. As will bedescribed in greater detail below, the techniques described hereinharness the advantageous qualities of both wireless data connections andminimize the inconvenience or risk introduced by each of theirdrawbacks.

The relationship established can optionally facilitate the automaticreconnection of the connection between a device and a peripheral displayunit; this automatic reconnection can optionally occur whenever thedevice and the peripheral display unit are within range of one anothersuch that the required data connection(s) may be established. The deviceand the peripheral display unit can optionally each store information onrespective local memories that is used to identify the other andsecurely and quickly reestablish the data connection with one another.Thus, it may be said that the device and the peripheral display unitestablish a “stored” relationship, such that they “remember” oneanother, and can automatically reconnect with one another.

Attention is now directed to user interfaces for an exemplary techniquefor establishing, via a wired-setup process, a relationship betweendevice 600 and peripheral display unit 610. FIG. 6B depicts a userinterface for establishing a relationship between device 600 andperipheral display unit 610 over a wired data connection, namely wiredsetup user interface 620. Wired setup user interface 620 can optionallybe displayed on display 608 in response to device 600 receiving a signalfrom peripheral display unit 610 over wired interface 606. That is, whenthe device 600 is connected to peripheral display unit 610, for examplefor the first time, peripheral display unit 610 can optionally send asignal to the device to indicate that the peripheral display unit 610 iscapable of establishing a relationship. In response to this signal overthe wired data connection, device 600 displays wired setup userinterface 620.

In some embodiments, wired setup user interface 620 provides variousoptions for a user, including wireless relationship affordance 622,wired relationship affordance 624, and charge-only affordance 626.Wireless relationship affordance 622 corresponds to authorization by theuser for the device 600 to establish a wireless connection with theperipheral display unit 610. Wireless relationship affordance 622 canoptionally also correspond to authorization for the device 600 toestablish a wired connection in addition to a wireless connection, as awired connection can optionally be understood as a lesser and includedpermission. Wired relationship affordance 624 corresponds toauthorization by the user for the device 600 to establish a wiredconnection with the peripheral display unit 610, without authorizingwireless connection. Finally, charge-only affordance 626 corresponds tothe denial of authorization by a user for the device 600 to establishany connection with the peripheral display unit 610; rather, the device600 will use the wired physical connection simply to charge its battery,and will not establish any connection via either a wired or wirelessconnection.

As will be described in greater detail below, selection of any of thethree affordances in wired setup user interface 620 can optionallyaffect the behavior of device 600 in both the immediate future and atvarious instances thereafter. That is, in some embodiments, device 600establishes the authorized connection in accordance with the selectedaffordance immediately, and also stores information in a local memorysuch that device 600 can optionally recall the user's selection at afuture time and establish only the authorized type of connection (wiredor wireless) in the future. Furthermore, device 600 can optionallydetermine whether, upon future connection of the wired data connection,to prompt the user to establish a relationship. In some embodiments,once the user has made a selection at wired setup user interface 620,interface 620 will not be automatically displayed again. In otherembodiments, wired setup user interface 620 can optionally beredisplayed upon some or all future instances of the wired physicalconnection being established.

FIG. 6B further depicts a user selection of wireless relationshipaffordance 622. In the example shown, user input 628 is a tap or touchinput at a location corresponding to affordance 622. However, otherinput methods (e.g., using a selection knob or joystick control) are, insome embodiments, used to make the selection instead.

FIG. 6C further depicts wired setup user interface 620. In particular,FIG. 6C depicts Bluetooth enablement prompt 630. Bluetooth enablementprompt 630 is a prompt provided to the user, such as by displaying ondisplay 608, indicating that the Bluetooth communication interface,which can optionally be second wireless communication interface 604,needs to be enabled. This prompt may be necessary during wired setup ofa wireless relationship, because various wireless data connections, suchas Bluetooth and Wi-Fi, may be necessary to establish the relationshipand connect the connection. Thus, if a user begins the setup process byphysically connecting device 600 and peripheral display unit 610 via awired data connection, the device 600 may need to activate any disabledwireless communication functionalities in order to complete setup of awireless relationship. In other embodiments not depicted in the figures,the wired setup user interface 620 includes a Wi-Fi enablement promptthat indicates that Wi-Fi communication, which can optionally correspondto first wireless communication interface 602, needs to be enabled.

Attention is now directed to user interfaces for an exemplary techniquefor wirelessly establishing (e.g., via a wireless setup) a relationshipbetween device 600 and peripheral display unit 610. FIG. 6D depicts auser interface for establishing a relationship between device 600 andperipheral display unit 610 over one or more wireless data connections,namely CarPlay settings menu 632. CarPlay settings menu 632 includesvarious setting and options pertaining to relationships that have beenor may be established with various vehicles. As shown in the interfaceunder the words “My Cars,” a stored relationship has already beenestablished between the device and a Volvo vehicle. As shown in theinterface under the words “Other Cars,” the device 600 is detecting twovehicles with which a stored relationship has not been established, aMercedes Benz vehicle and a Hyundai vehicle. In the example depicted,the names of the “other cars” each constitute affordances, such that thearea corresponding to the words “Mercedes Benz C-Class” constitutesavailable car affordance 634. Available car affordance 634 correspondsto an instruction by the user to wirelessly establish a storedrelationship between device 600 and peripheral display unit 610represented by the available car affordance 634. Techniques forestablishing this relationship will be explained in greater detailbelow. FIG. 6D further depicts user input 635, which corresponds toselection of available car affordance 634. In the example shown, theuser selection is a tap or touch input at a location corresponding toaffordance 634. However, other input methods (e.g., using a selectionknob or joystick control) are, in some embodiments, used to make theselection instead.

FIG. 6E depicts another user interface for wirelessly establishing arelationship between device 600 and peripheral display unit 610, namelyBluetooth settings menu 636. Bluetooth settings menu 636 includesvarious setting and options pertaining to relationships that have beenor may be established with various Bluetooth devices. As shown in theinterface under the words “My Devices,” a stored relationship hasalready been established between the device 600 and a home stereoBluetooth device. As shown in the interface under the words “OtherDevices,” the device 600 is detecting two Bluetooth devices with which astored relationship has not been established: a Mercedes Benz vehicleand a pair of wireless headphones. In the example depicted, the names ofthe “other devices” each constitute affordances, such that the areacorresponding to the words “Mercedes Benz C-Class” constitutes availableBluetooth device affordance 637. Available Bluetooth device affordance637 corresponds to an instruction by the user to wirelessly establish astored relationship between the device 600 and the peripheral displayunit 610 represented by the available Bluetooth device affordance 637.Techniques for establishing this relationship will be explained ingreater detail below. FIG. 6E further depicts user input 639, whichcorresponds to selection of available Bluetooth device affordance 637.In the example shown, the user selection is a tap or touch input at alocation corresponding to affordance 637.

FIG. 6F depicts another user interface for wirelessly establishing arelationship between device 600 and peripheral display unit 610, namelya wireless setup authorization interface 638. Wireless setupauthorization interface 638 can optionally be displayed in response tothe selection of either affordance 634 or 637 as described with respectto FIGS. 6D and 6E. Wireless setup authorization interface 638 issimilar in many respects to the interface depicted in FIG. 6B, which waspart of the interface for wired setup. In particular, wireless setupauthorization interface 638 includes wireless relationship affordance622 and wired relationship affordance 624, as the interface in FIG. 6Bdoes. As in FIG. 6B, wireless relationship affordance 622 corresponds toauthorization by the user for device 600 to establish a wirelessconnection with peripheral display unit 610 (and can optionally alsocorrespond to authorization for device 600 to establish a wiredconnection in addition to a wireless connection), while wiredrelationship affordance 624 corresponds to authorization by the user fordevice 600 to establish a wired connection with peripheral display unit610. Unlike the interface in FIG. 6B, wireless setup authorizationinterface 638 does not include an option to establish a charge-onlyrelationship, because the user has specifically sought out the Bluetoothor CarPlay menus described above with respect to FIGS. 6D and 6E inorder to reach authorization interface 638. That is, device 600 assumesthat the user does indeed want to establish a stored relationship, andseeks only to determine whether a wireless or wired-only relationshipshould be established.

In some embodiments not depicted in the figures, wireless setupauthorization interface 638 presents different options in accordancewith the manner in which the user accessed the authorization interface.For example, the interface can optionally provide the options forwireless and wired-only connections when a user accessed theauthorization interface 638 through CarPlay menu 632, but can optionallyprovide only the option for a wireless connection when the user accessedthe authorization interface 638 through Bluetooth menu 636 (as Bluetoothmenu 636 is inherently associated with wireless connections).

FIG. 6F further depicts user input 640 of wireless relationshipaffordance 622. In the example shown, the user selection is a tap ortouch input at a location corresponding to affordance 622; however,other input methods (e.g., using a selection knob or joystick control)are, in some embodiments, used to make the selection instead.

FIG. 6G depicts another user interface for wirelessly establishing arelationship between device 600 and peripheral display unit 610, namelyBluetooth credentials prompt 642. Bluetooth credentials prompt 642 is aninterface that prompts the user to enter Bluetooth credentials, such asa numerical code. These credentials can optionally be provided, forexample, on the display of the peripheral display unit 610. Thecredentials are displayed on the peripheral display unit 610 and enteredon the device 600 with which a relationship is being established. Thisprocess is designed to enhance security by ensuring that the user hascontrol over both the device 600 and the peripheral display unit 610. Insome embodiments, Bluetooth credentials prompt 642 is displayed only asa part of the wireless setup process, and not as a part of the wiredsetup process, because the presence of a wired physical connectionbetween the device 600 and the peripheral display unit 610 during wiredsetup adequately ensures that the user is legitimately in control ofboth device 600 and peripheral display unit 610.

FIG. 6H depicts a user interface for establishing, via a wired orwireless connection, a relationship between device 600 and peripheraldisplay unit 610. Namely, FIG. 6H depicts a contact-syncing prompt 644that provides information to the user about whether to sync contactsfrom device 600, such as a contacts list from a smartphone, withperipheral display unit 610. In some embodiments, contact-syncing prompt644 informs a user that the relationship being established does notrequire syncing of contacts, and that the relationship will functionproperly without syncing contacts. That is, while users may be used toknown methods of connecting their devices to peripheral display unitsincluding vehicles, such as by Bluetooth connections using the PhoneBook Access Profile (PBAP) specification, the current disclosureprovides embodiments in which syncing contacts is not necessary. Rather,the contacts from device 600 can remain stored on device 600 and simplybe communicated to peripheral display unit 610 as needed, rather thanbeing synced with or stored on peripheral display unit 610. Thus, notsyncing contacts avoids an unnecessary security risk without reducingthe functionality of the user interface presented on the peripheraldisplay unit 610 in conjunction with the device 600. Accordingly,contact-syncing prompt 644 includes an option to not sync contacts,which is presented in the form of “don't sync” affordance 646. As shownby the fact that “don't sync” affordance 646 is bolded and located onthe right-hand side of the prompt (proximate to a right-handed user'sthumb), “don't sync” affordance 646 is, in some embodiments, the defaultoption.

Attention is now directed to an exemplary user interface for accessing anative function of device 100 after a connection has been establishedwith peripheral display unit 610. FIG. 6I depicts device 100 andperipheral display unit 610 in a state in which a connection is active,such as when a wireless connection or a wired connection is active. FIG.6I further depicts detection of a user input, which, in the depictedexample is a depression 650 of hardware button 619. As will be describedin greater detail below, hardware button 619 can optionally beconfigured to perform various functions depending on the state ofperipheral display unit 610, whether a connection is active, and themanner in which button 619 is depressed. In the depicted example,depression 650 of the button 619 has activated personal digitalassistant interface 648 which is displayed on both display 608 anddisplay 618. In some other embodiments not depicted, personal digitalassistant interface 648 can optionally be provided only on display 618.

Attention is now directed to user interfaces for techniques forproviding battery-usage alerts in accordance with battery-usagepatterns. These techniques enable devices to provide battery-usagealerts that are not determined in accordance merely with the chargeamount of a device battery, but rather in accordance with battery-usagepatterns that can optionally provide useful historical and contextualinformation to the user, in order to allow for battery life to beoptimally conserved. In particular, these techniques may be useful indevices that are wirelessly connected (such as by in wirelessconnection) with a peripheral display unit, because such operation canbe battery-intensive. Finally, these techniques may be furtherparticularly useful in operating environments in which a device iswirelessly connected (such as by in wireless connection) to a peripheraldisplay unit in a vehicle, such as operating environment 500. In suchenvironments, these techniques enable battery-usage alerts provided inaccordance with information about a predefined or predicted destinationof the vehicle, and the predicted amount of time that will be requiredto arrive there. Thus, for example, these battery-usage notificationscan optionally warn a user if his battery is not going to last theduration of a trip while wirelessly connected to the vehicle.

FIG. 6J depicts an exemplary charge-level alert. In some embodiments, acharge-level alert is provided by one or both of device 600 andperipheral display unit 610 to which device 600 is connected by aconnection. In the example shown, charge-level alert 652 is displayed ondisplay 608. Charge-level alert 652 is displayed in accordance with adetermination by device 600 that the current battery charge level is 10%of the total battery charge capacity, and alert 652 conveys some or allof that information to the user.

FIG. 6K depicts an exemplary battery-usage alert. In some embodiments, abattery-usage alert is provided by one or both of device 600 andperipheral display unit 610 to which device 600 is connected by aconnection. Battery usage alerts, in some embodiments, are provided inaccordance with a determination by the device regarding a battery-usagepattern, which will be explained in greater detail below. In the exampleshown in FIG. 6K, battery-usage alert 654 is displayed on display 608.Battery-usage alert 654 is displayed in accordance with a determinationby device 600 that the battery has used 50% of its total charge capacitysince CarPlay (a wireless connection with a vehicle) was activated, andalert 654 conveys some or all of that information to the user.

FIG. 6L depicts another exemplary battery-usage alert. In the exampleshown, battery-usage alert 656 is displayed on both display 608 anddisplay 618. Battery-usage alert 656 is displayed in accordance with adetermination by device 600 that the battery is going to die beforedevice 600 reaches a destination. The manner in which such adetermination/prediction can optionally be made is discussed in greaterdetail below. Battery-usage alert 656 conveys some or all of thatdetermined information to the user.

FIG. 6M depicts an exemplary connection deactivation prompt. In someembodiments, a connection deactivation prompt is provided by one or bothof device 600 and peripheral display unit 610 to which device 600 isconnected by a connection. Connection deactivation prompts, in someembodiments, are provided in accordance with a determination by device600 regarding a charge level and/or a batter-usage pattern, as will beexplained in greater detail below. In the depicted example, connectiondeactivation prompt 658 is displayed on both display 608 and display618. In the example shown, connection deactivation prompt 658 prompts auser to deactivate CarPlay, as the battery level of device 600 is verylow. As CarPlay wireless connection can optionally be considered ahigh-power operation mode, connection deactivation prompt 658 encouragesthe user to deactivate CarPlay to conserve the small remaining batterycharge. FIG. 6M further depicts selection of an option to deactivateCarPlay in accordance with the connection deactivation prompt 658. Inthe depicted example, the selection is user input 660-a or 660-b, whichis a tap or touch user input detected on display 608 or display 618 at alocation corresponding to displayed affordances for deactivatingCarPlay. However, other input methods (e.g., using a selection knob orjoystick control) are, in some embodiments, used to make the selectioninstead.

Attention is now directed to exemplary user interfaces, wherein the userinterface at the device allows configuration of a user interface of aperipheral display unit. That is, in FIGS. 6N-6V, both device 600 andperipheral display unit 610 are depicted. In each of the figures,peripheral display unit 610 is displaying on display 618 user interface662, which in some embodiments is the primary user interface forperipheral display unit 610. User interface 662 comprises user interfaceobjects such as user interface object 663, which can optionally be anaffordance or icon for activating a corresponding function.

In each of FIGS. 6N-6V, device 600 is displaying on display 608configuration interface 664. Configuration interface 664 compriseswindow 668, which in some embodiments is a representation of userinterface 662. As shown in FIG. 6N, window 668 is a visualrepresentation of user interface 662, containing user interface objectrepresentations, such as user interface object representation 669. Inthe depicted example in FIG. 6N, each of the user interface objectrepresentations in window 668 corresponds to a user interface object inuser interface 662. Configuration interface 664 additionally comprisesavailable object tray 670, which itself comprises available userinterface object representations 672-a and 672-b.

As will be explained in greater detail below, configuration interface664 allows for the configuration, via interface 664, of interface 662.It is important to note that, in some embodiments, configuration of userinterface 662 can optionally be completed while a connection is activebetween device 600 and peripheral display unit 610. In some examples,configuration interface 664 can optionally be used when no connection isactive. In some of those embodiments, peripheral display unit 610 doesnot display anything until the next time the relationship is established(as user interface 662 can optionally always be hosted on device 600 andserved to peripheral display unit 610 live). In those examples,peripheral display unit 610 would of course not display user interface662 at all when the relationship is not active. In the figures below,user interface 662 is illustrated as persistently displayed during allstages of configuration, for illustrative purposes.

FIGS. 6N-6P depict the addition of a user interface object to userinterface 662. In FIG. 6N, user input 674-1 is detected in the form of atouch contact detected at a location corresponding to available userinterface object representation 672-a. Device 600 detects movement ofinput 674-1 from its original position at representation 672-a in tray670 toward window 668.

In FIG. 6O, device 600 continues to detect the movement of user input674 until it reaches its final position at 674-2. The device detectslift-off of the touch contact at the position corresponding to 674-2. Inresponse to detecting user input 674, configuration interface 664displays available user interface object representation 672-a movingfrom its original position in tray 670 to its final position in window668. Configuration interface 664 can optionally display an animation ofsaid movement. The movement of representation icons, in someembodiments, tracks the movement of a moving/dragging user input such asuser input 674. In embodiments where a connection between device 600 andperipheral display unit 610 is not active, configuration interface 664can optionally be updated as described above before the instruction toupdate user interface 662 is transmitted to peripheral display unit 610.

In FIG. 6P, a configuration instruction based on the user input depictedin FIGS. 6N and 6O has been passed from device 600 to peripheral displayunit 610. This instruction can optionally be passed immediately uponreceiving the above-described user input in embodiments where aconnection is active, or later upon the reestablishment of a connectionbetween device 600 and peripheral display unit 610. In accordance withthe configuration instruction, user interface 662 is updated to displaynew user interface object 676, which corresponds to representation 672-ain window 668. In some embodiments, the user interface for theperipheral display unit 610 is generated on device 600 and pushed toperipheral display unit 610, and the new use interface object isdisplayed on the peripheral display unit 610 when an updated userinterface generated by device 600 is pushed to peripheral display unit610.

FIGS. 6Q-6S depict the removal of a user interface object from userinterface 662. In FIG. 6Q, deletion affordance 678 is displayed, in someembodiments, to indicate a user interface object (or its representationsin window 668) that can optionally be deleted. In the depicted example,user input 678 is detected in the form of a touch or tap contactdetected at a location corresponding to deletion affordance 678.

In FIG. 6R, in response to detecting user input 674, configurationinterface 664 displays the deletion/removal of user interface objectrepresentation 672-a from window 668, and displays that representation672-a has returned to tray 670. Configuration interface 664 canoptionally display an animation of said deletion (such as an animationof representation 672-a moving back to tray 670). In embodiments where aconnection between device 600 and peripheral display unit 610 is notactive, configuration interface 664 can optionally be updated asdescribed above before the instruction to update user interface 662 istransmitted to peripheral display unit 610.

In FIG. 6S, a configuration instruction based on the user input depictedin FIG. 6Q has been passed from device 600 to peripheral display unit610. This instruction can optionally be passed immediately uponreceiving the above-described user input in embodiments where aconnection is active, or later upon the reestablishment of a connectionbetween device 600 and peripheral display unit 610. In accordance withthe configuration instruction, user interface 662 is updated to nolonger display user interface object 676.

FIGS. 6T-6V depict the rearrangement of a user interface object in userinterface 662. In FIG. 6T, user input 682-1 is detected in the form of atouch contact detected at a location corresponding to user interfaceobject representation 672-a. Device 600 detects movement of input 682-1from its original position at representation 672-a in window 670 towarda new position to the right.

In FIG. 6U, device 600 continues to detect the movement of user input682 until it reaches its final position at 682-2. The device detectsliftoff of the touch contact at the position corresponding to 682-2. Inresponse to detecting user input 682, configuration interface 664displays user interface object representation 672-a moving from itsoriginal position in window 668 to its final position further to theright in window 668. Configuration interface 664 can optionally displayan animation of said movement. The movement of the representation icons,in some embodiments, tracks the movement of a moving/dragging user inputsuch as user input 682. In embodiments where a connection between device600 and peripheral display unit 610 is not active, configurationinterface 664 can optionally be updated as described above before theinstruction to update user interface 662 is transmitted to peripheraldisplay unit 610.

In FIG. 6V, a configuration instruction based on the user input depictedin FIGS. 6T and 6U has been passed from device 600 to peripheral displayunit 610. This instruction can optionally be passed immediately uponreceiving the above-described user input in embodiments where aconnection is active, or later upon the reestablishment of a connectionbetween device 600 and peripheral display unit 610. In accordance withthe configuration instruction, user interface 662 is updated to displaythe repositioning of user interface object 676, which corresponds to therepositioning of representation 672-a in window 668.

Attention is now directed to FIG. 6W, which displays exemplary userinterfaces and configuration user interfaces for peripheral display unit610-b, which is different from the peripheral display unit 610 depictedin FIGS. 6N-6V. As shown in FIG. 6W, peripheral display unit 610-b hasdisplay 618-b that is of a different size and different shape thandisplay 618. In some embodiments, display 618-b is also of a differentresolution than display 618. Peripheral display unit 610-b has userinterface 684 that is different from user interface 662. User interface684 contains user interface objects such as user interface object 688.The user interface objects on user interface 684 can optionally bedifferent in identity, function, size, composition, arrangement, andappearance from the user interface objects on user interface 662.

FIG. 6W depicts that configuration interface 664, in some examples,further comprises second window 686, which is a representation of userinterface 684. In accordance with the differences of user interface 684,window 686 is also different from previously described window 668,containing user interface object representations, such as representation692, that correspond to the user interface objects of user interface684. In FIG. 6W, tray 670 contains available user interface iconrepresentations 690-a, 690-b, and 690-c, which are different from theavailable user interface icon representations in FIGS. 6N-6V.Accordingly, configuration interface 670 can optionally includedifferent available user interface object representations in tray 670 inaccordance with which user interface is being configured.

FIGS. 7A-7H are flow diagrams illustrating exemplary process 700 forestablishing a relationship between a device and a peripheral displayunit in accordance with some embodiments, such as those described abovewith reference to FIGS. 6A-6I. Process 700 is carried out by device 500(FIG. 5), which can optionally be electronic device 100 or 300 (FIGS. 1Aand 3A), in some embodiments.

Attention is now directed to FIG. 7A, which is a flow diagramillustrating an exemplary process for establishing a relationshipbetween a device and a peripheral display unit.

At block 702, the device receives a request from a user to authorize arelationship that corresponds to a connection between the device and theperipheral display unit, wherein the connection is a connection over afirst data connection with the peripheral display unit. An exemplaryperipheral display unit is peripheral display unit 610 of FIG. 6A. Insome embodiments, the first data connection is a wireless dataconnection, such as a Wi-Fi data connection. An exemplary Wi-Fi dataconnection is a connection formed between first wireless data interface602 and first wireless data interface 612 in FIG. 6A.

In some embodiments, the relationship is a stored relationship betweenthe device and the peripheral display unit, such that the device and/orperipheral display unit store data, on a local memory, indicating that aconnection with the other is authorized. The device and the peripheraldisplay unit may be said to “remember” one another. The relationship, asexplained below, can optionally permit automatic reconnection of theauthorized data connection at future points in time, such as wheneverthe device and peripheral display unit are within range of theauthorized data connection (e.g., when a wireless connection is able toconnect, or when a wired physical connection is plugged in). Therelationship can optionally authorize the exchange of phone-call data,user interface data, web-browsing data, media data, video stream data,navigation data, etc., with or without user input. In some embodiments,the relationship is a between a mobile phone and a motor vehicle, suchas a CarPlay-enabled vehicle.

An exemplary request to authorize a relationship that corresponds to aconnection over the first data connection is user input 628 in FIG. 6B,which can optionally be a touch or tap input detected by a touch screen.In some embodiments, the user selects “Authorize Wireless CarPlay” toauthorize a CarPlay relationship corresponding to a Wi-Fi connectionbetween the device and the peripheral display unit, which is integratedinto a vehicle.

At block 704, in response to receiving the request to authorize arelationship, the device establishes a relationship with the peripheraldisplay unit, wherein establishing the relationship includes receivingauthentication information from the peripheral display unit via a seconddata connection that is different from the first data connection. Insome embodiments, the second data connection is a wired data connection,such as a USB data connection. An exemplary wired data connection is thewired data connection between wired data interface 606 and wired datainterface 616 in FIG. 6A. In some embodiments, the second dataconnection is a wireless data connection, such as a Bluetooth dataconnection. An exemplary Bluetooth data connection is the Bluetooth dataconnection between second wireless data interface 604 and secondwireless data interface 614 in FIG. 6A. In some embodiments,establishing the relationship between the device and the peripheraldisplay unit includes receiving authentication information over thesecond data connection, such as a USB data connection or a Bluetoothdata connection, from the peripheral display unit. In some embodiments,the authentication information includes credentials, such as Wi-Ficredentials, for establishing the first data connection and forconnecting the first data connection immediately and/or in the future.

Blocks 706-710 occur while a connection between the device and theperipheral display unit over the first data connection is not active.That is, the relationship corresponding to the connection over the firstdata connection can optionally be established, but the actual dataconnection itself is not active. For example, for a device havingestablished a stored relationship with a peripheral display unit in avehicle, the device may be out of range of the vehicle such that theWi-Fi connection between the device and the peripheral display unit isnot currently connected.

At block 708, the device detects that the peripheral display unit isavailable for establishment of a connection. Establishment of aconnection may refer to the establishment of the first data connectionreferred to above. For example, in some embodiments, this detectionoccurs when the device is brought within range of the peripheral displayunit such that one or more of the wireless data interfaces in thedevice, such as wireless data interface 602 and 604, detects thepresence of the peripheral display unit. In some other embodiments, thisdetection occurs when the device is connected to the peripheral displayunit via a wired data connection, such as one supported by wired datainterface 606.

At block 710, in response to detecting that the peripheral display unitis available for establishment of a connection, the device establishes aconnection between the device and the peripheral display unit, whereinthe connection is a connection over the first data connection, whereinestablishing the connection between the device and the peripheraldisplay unit comprises providing the authentication information to theperipheral display unit to establish the connection. Thus, in someembodiments, to establish an active connection, the device provides theearlier-shared authentication information back to the peripheral displayunit. This allows the peripheral display unit to authenticate, viachecking the credentials, that the device is approved to form aconnection with the peripheral display unit. For example, in someembodiments, while the Wi-Fi connection referred to above is notactively connected (e.g., by a connection), in response to the devicecoming within range of the peripheral display unit and detecting (viaone or more wireless data connections such as Bluetooth and Wi-Fi) thatthe peripheral display unit is available for the establishment of aconnection, the device establishes a Wi-Fi connection with theperipheral display unit. The process of establishing said Wi-Ficonnection includes, in some embodiments, sending the Wi-Fi credentialsback to the peripheral display unit, thereby proving that a trustedrelationship was previously formed.

Attention is now directed to FIGS. 7B-7C, which depict flow diagramsillustrating an exemplary process for establishing a relationshipbetween a device and a peripheral display unit via a wired connection.As explained above with reference to FIG. 6, care should be taken todistinguish between the connection itself (e.g., the connection) beingwired or wireless, versus the method for establishing the relationshipbeing wired or wireless. FIGS. 7B and 7C represent an exemplary methodin which the method for establishing the relationship is wired. Forexample, the relationship can optionally be established over a wiredphysical connection such as a USB wire, which can optionally carry datacreating a wired data connection. FIGS. 7B and 7C depict method 702-a,which is an optional process performed as a part of block 702. That is,the steps in method 702-a primarily expand on the process of receiving arequest to establish a relationship, as explained above with referenceto block 702.

At block 712, optionally, the second data connection (of block 704) is awired data connection. An exemplary wired data connection is the wireddata connection between wired data interface 606 and wired datainterface 612 in FIG. 6A. In some embodiments, the wired data connectionis a data connection between a USB connector, such as one integratedinto a peripheral display unit in a vehicle, and a micro-USB connectoror a Lightning connector, such as one integrated into a mobile device.

Blocks 714-724 occur while a relationship has not been established withthe peripheral display unit. That is, in some embodiments, these stepsoccur before the device and the peripheral display unit have exchangedcredentials with one another or stored data reflecting a relationshipwith one another. In some embodiments, these steps occur upon the firsttime the device and the peripheral display unit come into connection orcommunication with one another, such as after a user purchases a newdevice (e.g., a new phone) or a new peripheral display unit (e.g., a newdata-connection-enabled system).

At block 714, optionally, the device receives a communication from theperipheral display unit over the wired data connection between thedevice and the peripheral display unit. The communication received bythe device can optionally be a message indicating that the peripheraldisplay unit is capable of establishing a relationship with the device,such as a message indicating that the peripheral display unit isCarPlay-enabled. The message can optionally further indicate that norelationship has yet been established between the device and theperipheral display unit. In some embodiments, the user connects thedevice to the peripheral display unit via a wired connection (such as aUSB connection), possibly for charging, and, upon connecting, the deviceand the peripheral display unit can optionally automatically exchangemessages, including the communication from the peripheral display unitto the device explained above.

At block 718, optionally, in response to receiving the communication,the device displays a first user interface prompting the user toauthorize a relationship with the peripheral display unit. An exemplaryuser interface prompting the user to authorize a stored relationshipwith the peripheral display unit is user interface 620 in FIG. 6B.

At block 720, optionally, the user interface includes a first optioncorresponding to authorizing a wireless relationship between the deviceand the peripheral display unit. Selection of this option can optionallycause the relationship established to be one that allows for a wirelessconnection, including a wireless connection established immediately andwireless connections automatically established at future times. Thus,despite the setup process being carried out over a wired connection, thedevice can optionally establish a wireless relationship that allows theexchange of content over a wireless data connection, such as the Wi-Firelationship described above. An exemplary first option is wirelessrelationship affordance 622 in FIG. 6B.

In some embodiments, selection of this option can optionally allow forthe establishment of a wired relationship (as described below) inaddition to a wireless relationship. This is because a wiredrelationship can optionally be considered a lesser-and-includedauthorization, as it may be a more secure connection, a more reliableconnection, and/or a less battery-intensive connection. In someembodiments, the device would be authorized to connect to the peripheraldisplay unit via either a wireless connection (e.g., Wi-Fi) or a wiredconnection (e.g., USB). For example, if, at a future time, the wirelessdata interface of the device were disabled, a connection via a wiredconnection could still be established if the device was connected to theperipheral display unit.

At block 722, optionally, the user interface includes a second optioncorresponding to authorizing a wired relationship between the device andthe peripheral display unit. Selection of this option can optionallycause the relationship established to be one that allows for a wiredconnection, including a wired connection established immediately andwired connections automatically established at future times. In someembodiments, the device would thus not be authorized to automaticallyconnect wirelessly to the peripheral display unit, but it would beauthorized to automatically connect via a wired connection to theperipheral display unit. Therefore, in some embodiments, when the userplugs in the device to the peripheral display unit (e.g., by chargingover a USB connection), the device can optionally automaticallyestablish a data connection (e.g., a connection) to the peripheraldisplay unit for sharing content. An exemplary second option is wiredrelationship affordance 624 in FIG. 6B.

In some embodiments, selection of this second option can optionallyprevent the device from prompting the user to authorize a wirelessrelationship upon future instances of the device being connected to theperipheral display unit via a wired connection.

At block 724, optionally, the user interface includes a third optioncorresponding to authorizing a charging-only connection between thedevice and the peripheral display unit. In some embodiments, byselecting this third option, the user denies permission to the device toauthorize either a wireless or a wired relationship. Thus, in someembodiments, the device does not connect to the peripheral display uniteither immediately or in the future (without subsequent userintervention) for the exchange of content. Rather, in some embodiments,the wireless connection is used only to charge the device's battery, notto exchange content. In some embodiments, selection of this third optioncan optionally prevent the device from prompting the user to authorize arelationship upon future instances of the device being connected to theperipheral display unit via a wired connection. An exemplary thirdoption is charge-only affordance 626 in FIG. 6B.

At block 726, optionally, the request from a user to establish arelationship with the peripheral display unit comprises selection of thefirst option. In some embodiments, selection of the first optioncomprises selection of an affordance representing the first option. Insome embodiments, selection of the first option comprises detection of atouch input at a location corresponding to the first option. Anexemplary selection of the first option is user input 628 in FIG. 6B,which in some embodiments can optionally be a touch or tap input on atouch screen. In some embodiments, a user selecting the first option inthe displayed user interface to authorize a wireless data connectionconstitutes, at least in part, the request to establish a relationshipwith the peripheral display unit.

At block 728, optionally, the authentication information (of block 704)comprises credentials for a wireless data connection distinct from thefirst data connection. An exemplary wireless data connection distinctfrom the first data connection is a data connection between secondwireless data interface 604 and second wireless data interface 614 inFIG. 6A, such as short-range, low-power, and/or low-bandwidth wirelessdata connection (e.g., a Bluetooth data connection). In someembodiments, the first data connection is high-power, high-bandwidth,and/or long-range wireless data connection (e.g., a Wi-Fi dataconnection) and the wireless data connection distinct from the firstdata connection is a Bluetooth data connection. In some embodiments,Bluetooth credentials are shared between the peripheral display unit andthe device over the wired connection during the setup process, such thatthe wireless data connection distinct from the first data connection canbe automatically authenticated and connected immediately and/or atfuture instances, without user intervention. As will be explained below,the wireless data connection distinct from the first data connection canbe used to facilitate quick and efficient automatic reconnection of thefirst data connection, particularly when the first data connection is awireless data connection.

It should be noted that such exchange of wireless data credentials suchas Bluetooth credentials is, in some embodiments, not doneautomatically, and can require user input and authorization, such as theuser typing in a Bluetooth credentials code. However, in someembodiments, the credentials are automatically shared via the wiredconnection because the presence of a wired physical connection serves asadequate indication that the user is in physical control, and thereforelegitimate possession and control, of both the device and the peripheraldisplay unit.

In some embodiments, in response to receiving a request to authorize awireless relationship, the device prompts the user to activate adeactivated wireless data interface. For example, in some embodiments,when a user elects to set up a wireless relationship, the device canoptionally prompt a user to activate an inactive Wi-Fi or Bluetooth datainterface. This functionality is particularly relevant in the context ofthe wired setup process, as the user can optionally begin the wiredsetup process with some or all wireless data interfaces disabled. Anexemplary prompt to activate a wireless data interface is Bluetoothenablement prompt 630 in FIG. 6C.

Attention is now directed to FIGS. 7D and 7E, which depict flow diagramsillustrating an exemplary process for establishing a relationshipbetween a device and a peripheral display unit via a wirelessconnection. As explained above with reference to FIG. 6, care should betaken to distinguish between the connection itself (e.g., therelationship) being wired or wireless, versus the method forestablishing the relationship being wired or wireless. FIGS. 7D and 7Erepresent an exemplary method in which the method for establishing therelationship is wireless. For example, the relationship can optionallybe established over a wireless connection such as a Bluetooth dataconnection. FIGS. 7D and 7E depict method 702-b, which is an optionalprocess performed as a part of block 702. That is, the steps in method702-b primarily expand on the process of receiving a request toestablish a relationship, as explained above with reference to block702.

At block 730, optionally, the second data connection (of block 704) is awireless data connection. An exemplary wireless data connection is thewireless data connection between second wireless data interface 604 andsecond wireless data interface 614 in FIG. 6A. In some embodiments, thewireless data connection is a Bluetooth data connection, such as betweena Bluetooth communication interface integrated into a peripheral displayunit in a vehicle, and a Bluetooth data connection integrated into amobile device.

Blocks 732-734 occur while a relationship has not been established withthe peripheral display unit. That is, in some embodiments, these stepsoccur before the device and the peripheral display unit have exchangedcredentials with one another or stored data reflecting a relationshipwith one another. In some embodiments, these steps can optionally occurupon the first time the device and the peripheral display unit come intoconnection or communication with one another, such as after a userpurchases a new device (e.g., a new phone) or a new peripheral displayunit (e.g., a new CarPlay-enabled system).

At block 734, optionally, the device detects, via one or more of thefirst and second data connections, that the peripheral display unit isavailable for the establishment of a relationship. In some embodiments,the device uses one or more wireless data connections to search forperipheral devices in proximity that are available for connection. Insome embodiments, the device exchanges data with detected peripherals todetermine what kind of peripheral it is, whether it is capable ofconnecting with the device, the name of the peripheral, and informationabout the security of a connection formed with the device (such aswhether connecting to the device would require a password or otherauthentication or credentials).

At block 736, optionally, in response to detecting that the peripheraldisplay unit is available for the establishment of a relationship, thedevice displays a fourth option for authorizing a relationship with theperipheral display unit. In some embodiments, the fourth optiondisplayed is an affordance indicating the name or identity of theavailable peripheral, because the device need not be physicallyconnected to a peripheral via a wired connection, the user may need anindication of what peripherals are available for wireless connection, soindication via the display of a name of an available peripheral isuseful.

At block 738, optionally, the fourth option for authorizing arelationship is an item in a list of Bluetooth-capable devices. Anexemplary fourth option that is an item in a list of Bluetooth-capabledevices is available Bluetooth device affordance 637 in Bluetoothsettings interface 636 in FIG. 6E.

At block 740, optionally, the fourth option for authorizing arelationship is an item in a list of vehicles with peripheral displayunits. An exemplary fourth option that is an item in a list of vehicleswith peripheral display units is available car affordance 634 in CarPlaysettings interface 632 in FIG. 6D.

In some embodiments, such as those depicted in FIGS. 6D and 6E, when thedevice detects that a peripheral display unit is available for theestablishment of a relationship, an affordance appears, such asaffordance 634 or 637, indicating the name of the peripheral displayunit and that the unit is available for establishing a relationship.Affordance 634 and affordance 637 indicate that a peripheral displayunit related to a Mercedes Benz is available for the establishment of arelationship, as indicated by the name of the vehicle appearing in an“other cars” or “other devices” list.

At blocks 742 and 744, optionally, the request from a user to establisha relationship comprises selection of the fourth option, and the devicedetects selection of the fourth option. In some embodiments, selectionof the fourth option comprises selection of an affordance representingthe fourth option. In some embodiments, selection of the fourth optioncomprises detection of a touch input at a location corresponding to thefourth option. An exemplary selection of the fourth option is user input635 in FIG. 6D, or user input 639 in FIG. 6E. Both inputs 635 and 639,in some embodiments, are a touch or tap input on a touch screen. In someembodiments, a user selecting the fourth option in the displayed userinterface to authorize a wireless data connection constitutes, at leastin part, the request to establish a relationship with the peripheraldisplay unit.

At block 746, optionally, in response to detecting selection of thefourth option, the device displays a second user interface prompting theuser to authorize a relationship with the peripheral display unit. Insome embodiments, after detecting selection of an option or affordanceindicating a device with which to wirelessly establish a relationship,the device provides a prompt inquiring as to which kind ofrelationship—wired or wireless—the user would like to wirelesslyestablish. An exemplary user interface prompting the user to authorize arelationship with the peripheral display unit is user interface 638 fromFIG. 6F.

At block 748, optionally, the second user interface comprises a fifthoption corresponding to authorizing a wireless relationship between thedevice and the peripheral display unit. In some embodiments, this fifthoption functions in the same way as the first option described abovewith reference to block 720. An exemplary fifth option is wirelessrelationship affordance 622 in FIG. 6F.

At block 750, optionally, the second user interface comprises a sixthoption corresponding to authorizing a wired relationship between thedevice and the peripheral display unit. In some embodiments, this sixthoption functions in the same way as the second option described abovewith reference to block 722. An exemplary sixth option is wiredrelationship affordance 624 in FIG. 6F.

Notably, in some embodiments, the second user interface for promptingthe user to authorize a relationship with the peripheral display unit,such as user interface 638, does not contain an option for a charge-onlyconnection, as interface 620 for wired-setup did. This is because, insome embodiments, user interface 638 is accessed through the deliberateselection of options to initiate a relationship-establishment process,such as the selection of affordance 634 or 637 by the user. Accordingly,it would seem unreasonable that, after selecting those options, the userwould desire to instruct the device to authorize a charge-onlyrelationship. Therefore, in some embodiments, the device determines,based on which user interfaces have recently been displayed or whichoptions have recently been selected by the user, which options are mostlikely to be selected by the user. Making such a determination, andchoosing to display only those options which are more likely to beselected by the user while suppressing display of options that are lesslikely to be selected, can increase the efficiency of the method andreduce the cognitive burden on the user in considering multiple options.Moreover, interface 638 is designed for use whether or not the device isphysically connected by a wired physical connection to the peripheraldisplay unit, so an option regarding a charging connection—which inalmost all cases requires a wired physical connection—could be unhelpfulto the user. For example, without a wired physical connection, thedevice would be unable to establish a wired data connection if the userwere to select such an option, so inconvenience, inefficiency, cognitivestrain, and confusion are avoided by suppressing display of an option toestablish a wired connection in some embodiments.

At block 752, optionally, the request from the user to establish arelationship with the peripheral display unit comprises selection of thefifth option. In some embodiments, selection of the fifth optioncomprises selection of an affordance representing the fifth option. Insome embodiments, selection of the fifth option comprises detection of atouch input at a location corresponding to the fifth option. Anexemplary selection of the fifth option is user input 640 in FIG. 6F.Input 640, in some embodiments, is a touch or tap input on a touchscreen. In some embodiments, a user selecting the fifth option in thedisplayed second user interface to authorize a wireless data connectionconstitutes, at least in part, the request to establish a relationshipwith the peripheral display unit.

At block 754, optionally, establishing a relationship with theperipheral display unit (of block 704) includes displaying on thedisplay of the device a third user interface prompting the user to entercredentials for the wireless data connection. In some embodiments, thedevice can optionally prompt the user to enter credentials correspondingto the wireless data connection by which the user is trying to establisha relationship with the peripheral display unit. For example, in someembodiments, the device can optionally prompt the user to enter aBluetooth confirmation code, such as a Bluetooth confirmation codedisplayed on the display of the peripheral display unit. This processcan optionally be in an attempt to establish that the user haslegitimate control over both the peripheral display unit and the device,and to prevent unauthorized users from wirelessly establishing arelationship with the peripheral display unit, such as from nearby theperipheral display unit. An exemplary interface prompting the user toenter credentials for the wireless data connection is user interface 642in FIG. 6G.

Attention is now directed to method 704-c in FIG. 7F. Method 704-c is anoptional process performed as part of block 704. That is, the steps inmethod 704-c primarily expand on the process of establishing arelationship with the peripheral display unit, as explained above withreference to block 704.

At block 756, optionally, establishing a relationship comprisesdisplaying a fourth user interface providing the user with an option toestablish a relationship with the peripheral display unit withoutsyncing device contacts with the peripheral display unit. In someembodiments, such a user interface is displayed after a request toauthorize a relationship, such as after a user selects wirelessrelationship affordance 622 in FIG. 6B or wireless relationshipaffordance 622 in FIG. 6F.

An exemplary user interface providing the user with an option toestablish a relationship with the peripheral display unit withoutsyncing device contacts is user interface 644 in FIG. 6H. In interface644, the user is explicitly prompted and encouraged not to synccontacts, rather than being encouraged and prompted to sync contacts.While syncing contacts (e.g., transferring contact information from amobile phone for storage in a memory of a peripheral accessory, such asa motor vehicle) has been the norm in the prior art, there is no need tosync contacts in accordance with the methods disclosed herein. Savingcontacts in a memory of a peripheral display unit creates an unnecessarysecurity risk by exposing contact data to other users of the peripheraldisplay device. For example, when a user of a system syncs his contactsto a Bluetooth-enabled vehicle, the contacts are, in many circumstances,thereafter accessible by other users of the car. However, by using awired or wireless relationship such as those disclosed herein, such asCarPlay, there is no need to store contacts on the peripheral displayunit. Rather, the peripheral display unit can simply use the connectionto access the contacts as needed while the connection is active, withoutstoring contact data on a memory at the peripheral display unit.

In accordance with these considerations, in some embodiments, notsyncing contacts is the default option. In some embodiments, such asinterface 644, the user is explicitly informed that the connection willwork without syncing contacts, and/or that the default option is to notsync contacts. For example, in user interface 644, the option to notsync contacts is highlighted by being displayed in bold font (and canoptionally be displayed in a different color). The option to not synccontacts is also displayed as a default in that it is located on theright side of the interface, proximate to the dominant thumb of mostusers of the device. Furthermore, the option to sync contacts isdisparaged by being presented as “Sync Anyway”; thus, even if a useronly reads the text on the buttons, the user will recognize that thesyncing option is disfavored.

Attention is now directed to method 704-d in FIG. 7F. Method 704-d is anoptional process performed as part of block 704. That is, the steps inmethod 704-d primarily expand on the process of establishing arelationship with the peripheral display unit, as explained above withreference to block 704.

At block 758, optionally, establishing a relationship with theperipheral display unit comprises storing an indication of whether toestablish a wireless connection, a wired connection, or a charge-onlyconnection. In some embodiments, data is stored locally on a memory ofthe device that indicates the preference that has been selected by theuser. For example, if the user elects a wireless relationship, then thedevice will save an indication of this preference so that it canautomatically wirelessly reconnect to the peripheral at future times.For example, if the user elects wired relationship, then the device willsave an indication of this preference so that it can reconnect to theperipheral at future times only when a wired data interface isconnected. For example, if the user elects a charge-only connection,then the device will save an indication of this preference so that, infuture instances of the device being able to connect with the peripheraldisplay unit, it will not connect or share content, but instead willestablish a charge-only connection. The stored indication, in someembodiments, also allows the device to not prompt the user to authorizea connection after the user has done so once, or, in some otherembodiments, to prompt the user periodically at future times after theuser has declined such authorization once. In some embodiments, anindication of whether to establish a connection (and of what kind) isalternately or additionally stored on a local memory of the peripheraldisplay unit.

Attention is now directed to method 710-e in FIG. 7F. Method 710-e is anoptional process performed as part of block 710. That is, the steps inmethod 710-e primarily expand on the process of establishing aconnection between the device and the peripheral display unit, asexplained above with reference to block 710.

At block 760, optionally, detecting that the peripheral display unit isavailable for establishment of a connection occurs while the device isin a locked state.

At block 762, optionally, establishing a connection between the deviceand the peripheral display unit occurs while the device remains in alocked state.

In some embodiments of blocks 760 and 762, a user carrying a deviceapproaches a peripheral display unit. For example, a user carrying amobile device in his pocket gets into a CarPlay-enabled car. The devicedetects that the peripheral is available for the establishment of aconnection, in some embodiments, while the device is in a locked state.Moreover, the device establishes a connection with the peripheral, insome embodiments, while the device remains in a locked state.Accordingly, in some embodiments, the user's device detects theauthorized peripheral display unit from the user's pocket andautomatically wirelessly reestablishes a connection, and the user is notrequired to take the device out of his pocket or physically access thedevice in order for the connection to be completely established. Thus,in some embodiments, the user can simply enter his car and begininteracting with the peripheral display unit, without having to unlock,plug in, or otherwise interact with the device. In some embodiments,this functionality improves driver convenience and driver safety.

In some embodiments, being in a locked state means that the device isnot displaying any output on its display. In some embodiments, being ina locked state means that the device is not responsive to touch inputsdetected on its touch-sensitive surface. In some embodiments, being in alocked state means that access to some functions is restricted. In someembodiments, being in a locked state means that the device is in alow-power mode. In some embodiments, being in a locked state means thatthe device requires authentication by a user, such as a user entering apasscode or submitting biometric (e.g., fingerprint) authentication,before the user can access some functions of the device.

Attention is now directed to method 710-f in FIG. 7G. Method 710-f is anoptional process performed as part of block 710. That is, the steps inmethod 710-f primarily expand on the process of establishing aconnection between the device and the peripheral display unit, asexplained above with reference to block 710.

At block 764, optionally, establishing the connection between the deviceand the peripheral display unit further comprises communicating with theperipheral display unit via a wireless data connection distinct from thefirst data connection, wherein a message communicated over the wirelessdata connection comprises an explicit instruction to establish aconnection via the first data connection. In some embodiments,particularly embodiments in which the first data connection (on whichthe connection will exist) is a wireless data connection, it can bebeneficial to use another wireless data connection to assist inreestablishing the first data connection. In some embodiments, themessage passed over the wireless data connection, comprising an explicitinstruction to reconnect via the first data connection, identifies thefirst data connection such that it can be located and authenticated morequickly. For example, when the first data connection is a Wi-Fi dataconnection, it can be beneficial to use a Bluetooth data connection tohelp reestablish the Wi-Fi data connection more quickly. In someembodiments, as a Bluetooth data connection reconnects more quickly thana Wi-Fi data connection, the reestablished Bluetooth data connection isused to hasten the reconnection of the Wi-Fi data connection byproviding an instruction to reconnect via the Wi-Fi data connection(including, in some embodiments, an identification of the identity ofthe available Wi-Fi connection). This process is optionally called“bootstrapping” the first data connection with the (other) wireless dataconnection (e.g., the Bluetooth connection).

At block 766, optionally, the instruction to establish a connection viathe first data connection is transferred from the peripheral displayunit to the device. In some embodiments, for example, the instruction toestablish a Wi-Fi connection is transferred over a Bluetooth connectionfrom a peripheral display unit to a device. For example, aCarPlay-enabled car can optionally transfer from the peripheral displayunit to a user's mobile phone, via Bluetooth, an explicit instruction toreconnect to the CarPlay Wi-Fi connection.

Attention is now directed to block 768 in FIG. 7G. Block 768 is part ofprocess 700 for establishing a relationship between a device and aperipheral display unit in accordance with some embodiments. Block 768optionally occurs following blocks 706-710 in FIG. 7B.

At block 768, optionally, while a connection between the device and theperipheral display unit over the first data connection is active, thedevice transfers content over the first data connection. In someembodiments, once the relationship is formed in steps 706-710, contentcan optionally then be transferred over the first data connection. Asexplained previously, the first data connection can optionally be awired data connection (such as a USB data connection), such as aconnection between wired data interface 606 and wired data interface 616in FIG. 6A, or it can optionally be a wireless data connection (such asa Wi-Fi data connection), such as a connection between first wirelessdata interface 602 and first wireless data interface 612 in FIG. 6A. Insome embodiments, content includes audio data, video data, phone calldata, navigation data, web-browsing data, media data, user interfacedata, user input data, and more. In some embodiments, as explained abovewith reference to the option to not sync contacts, content transferredover the connection includes data relating to stored contactinformation.

Attention is now directed to blocks 770-784 in FIG. 7H. Blocks 770-784are part of process 700 for establishing a relationship between a deviceand a peripheral display unit in accordance with some embodiments.Blocks 770-784 optionally occur following blocks 706-710 in FIG. 7B.

At block 770, optionally, the peripheral display unit has a discoverablestate and a non-discoverable state, and the peripheral display unitcomprises a hardware button configured to cause the peripheral displayunit to enter the discoverable state. In some embodiments, adiscoverable state is a state in which the peripheral display unit isconfigured to allow communication with and access by a device, for theformation of a stored relationship and the establishment of aconnection, while a non-discoverable state is a state where the deviceis configured to not allow such communication and access. Thus, a userof a device, in some embodiments, is only able to establish a storedrelationship and establish a connection when the peripheral display unitis in a discoverable state.

In some embodiments, a hardware button is provided on the peripheraldisplay unit. An exemplary hardware button is hardware button 619 inFIG. 6I. In some embodiments, the hardware button is a physical,actuable button or switch located in a housing of the peripheral displayunit. In some embodiments, the button is a button located on thesteering wheel of a vehicle in which the peripheral display unit isprovided. In some embodiments, the hardware button is a touch-sensitive,pressure-sensitive, presence-sensitive, and/or capacitive-sensingmechanism, such as a touch pad, touch surface, or touch screen.

In some embodiments, the hardware button is configured to cause theperipheral display unit to enter the discoverable state, such thatinteraction with the hardware button (e.g.,depression/actuation/contacting of the button) is configured to causethe peripheral display device to leave the non-discoverable state andenter the discoverable state (e.g., instead of requiring a user tonavigate through a complex series of settings menus to place theperipheral display device into the discoverable state).

At block 772, optionally, the hardware button is further configured toactivate a native function of the peripheral display unit unrelated tomaking the unit discoverable. In some embodiments, the native functionof the peripheral display unit is a voice-recognition function or avoice-activated interface inherent to an entertainment/media head unitof an accessory or peripheral display unit (such as a voice-activationfunction on a motor vehicle).

At block 774, optionally, the hardware button is further configured toactivate a function of the device distinct from the native function ofthe peripheral display unit and unrelated to making the peripheraldisplay unit discoverable. In some embodiments, the function of thedevice is a voice recognition function, such as a personal digitalassistant function that responds to the user's voice queries and voicecommands to perform various functions.

In some embodiments in which the hardware button is configured toperform various unrelated functions, the button performs differentfunctions depending on the state of the peripheral display unit and thedevice. For example, the hardware button can optionally performdifferent functions depending on whether the peripheral display unit isin a discoverable or non-discoverable state, and it can optionallyperform different functions depending on whether the device is activelyconnected by a connection with the peripheral display unit. Furthermore,in some embodiments, the button performs different functions dependingon a characteristic of the selection of the button. For example, thebutton can optionally perform different functions depending on whetherit is selected with a short press (lasting less than a predefined amountof time) or a long press (lasting more than a predefined amount oftime).

At block 776, optionally, a short press of the hardware button activatesa native voice recognition function of the peripheral display unit, anda long press of the hardware button causes the peripheral display unitto enter the discoverable state. An exemplary press of the hardwarebutton is shown by depression 650 of button 619 in FIG. 6I, which canoptionally be a short or long press in some embodiments. In someembodiments, the peripheral display unit and the device are notcurrently actively connected by a connection with one another, and ashort press of the hardware button causes the peripheral to activate anative voice recognition function, while a long press of the hardwarebutton causes the device to enter a discoverable state. In someembodiments, if the device is already in a discoverable state, a longpress has no effect; in other such embodiments, a long press causes thedevice to leave the discoverable state. In some embodiments, such asembodiments in which the peripheral display unit is already activelyconnected by a connection to a device, the hardware button performsadditional functions.

Blocks 778-784 occur while a connection between the device and theperipheral display unit over the first data connection is active. Forexample, in some embodiments, these steps occur after the wired orwireless setup processes and the connection processes explained abovehave occurred. In some embodiments, these steps occur while arelationship is active and the device is sharing content with theperipheral display unit, such as in a CarPlay connection.

At block 780, optionally, the device detects a long press of thehardware button. An exemplary long press of the hardware button isdepression 650 of button 619 in FIG. 6I.

At block 782, optionally, in response to detecting the long press of thehardware button, the device activates the function of the device. Insome embodiments, the peripheral detects a long press of the hardwarebutton. Upon determining that a connection is active at the time, theperipheral passes data regarding the long press to the device. Inresponse to receiving indication of the long press, the device activatesa function of the device, in some embodiments.

At block 784, optionally, the function of the device is a voicerecognition function. In some embodiments, the function of the device isa personal digital assistant function that responds to the user's voicequeries and voice commands to perform various functions. An exemplarypersonal digital assistant function is depicted by personal digitalassistant interface 648 in FIG. 6I. In some embodiments, activation of apersonal digital assistant comprises display and/or audio output by oneor both of the device and the peripheral display unit. In someembodiments, a microphone of the device and/or a microphone of theperipheral display unit are activated, in accordance with the activationof the personal digital assistant, to listen for voice commands orqueries.

In some embodiments, the device displays an option to activate ordeactivate connection authorization. In some embodiments, this optionallows a user to elect to “turn off” the connection authorization, suchthat the device will not reestablish or reconnect to a peripheraldisplay unit, even if a trust relationship has already been establishedwith that peripheral display unit. In some embodiments, if theconnection is active, turning off the connection authorization willcause the connection to deactivate. In some embodiments, the option toactivate or deactivate the connection authorization is displayed as aswitch or affordance. In some embodiments, the device presents a list oftrusted peripheral display units, and when the user selects a unit fromthe list, the device presents the option to activate or deactivate theconnection authorization. In some embodiments in which the device hasestablished a relationship with multiple peripheral display units,connection authorizations can optionally be activated and deactivatedfor any of the peripheral display units independently of the others.

In some embodiments, the device displays an option to terminate arelationship with a peripheral display unit. In some embodiments, thisoption allows a user to elect to “forget” a trusted peripheral displayunit, such that the peripheral display unit will no longer be consideredtrusted, and no connection will automatically reestablish with theperipheral display unit. In some embodiments, if the connection isactive, terminating the relationship will cause the connection todeactivate. In some embodiments, the option to terminate therelationship is displayed as an affordance. In some embodiments, thedevice presents a list of trusted peripheral display units, and when theuser selects a unit from the list, the device presents the option toterminate the relationship with the peripheral display unit. In someembodiments in which the device has established a relationship withmultiple peripheral display units, relationships can optionally beterminated for any of the peripheral display units independently of theothers.

In some embodiments, when a connection is active, the device andperipheral display unit both rely on a cellular connection of thedevice, such that data is passed to and from the internet through thedevice's cellular connection, before or after being shared over theconnection between the device and the peripheral. In some embodiments,the opposite is true, in that the device and peripheral both rely on acellular connection of the peripheral display unit. In some embodiments,the user can optionally manually select (via an interface presented atthe peripheral display unit and/or the device) which cellular connectionto use. In some embodiments, an indicator/warning is displayed by thedevice and/or the peripheral display unit when the user is using onecellular connection or the other. Such a warning can be useful to userswho may not be accustomed to choosing between two different cellularconnections, possibly provided by different carriers, at differentconnection speeds, for different rates/costs, and/or with different datarestrictions and limits. In some embodiments, for example, theperipheral display unit displays an “LTE” indicator when the cellularconnection of the device is being used, and does not display an “LTE”indicator when the cellular connection of the peripheral display unit isbeing used.

It should be understood that the particular order in which theoperations in FIGS. 7A-7H have been described is merely exemplary and isnot intended to indicate that the described order is the only order inwhich the operations could be performed. One of ordinary skill in theart would recognize various ways to reorder the operations describedherein. Additionally, it should be noted that details of other processesdescribed herein with respect to other methods described herein (e.g.,methods 800, 900) are also applicable in an analogous manner to 700described above with respect to FIGS. 7A-7H. For example, the devices,peripheral display units, relationships, connections, and userinterfaces described above with reference to 700 optionally have one ormore of the characteristics of the devices, peripheral display units,relationships, connections, and user interfaces described herein withreference to other methods described herein (e.g., methods 800, 900).

FIGS. 8A-8D are flow diagrams illustrating exemplary process 800 formonitoring battery-usage patterns and providing alerts in accordancetherewith, in accordance with some embodiments such as those describedabove with reference to FIGS. 6J-6M. Process 800 is carried out bydevice 500 (FIG. 5), which can optionally be electronic device 100 or300 (FIGS. 1A and 3A), in some embodiments.

Attention is now directed to FIG. 8A, which is a flow diagramillustrating an exemplary process for providing charge-level alerts andproviding battery-usage alerts.

At block 802, a device monitors battery usage of the device, whereinmonitoring battery usage of the device includes monitoring a chargelevel of one or more batteries of the device and monitoring batteryusage patterns of the one or more batteries of the device. An exemplarydevice is device 600 of FIG. 6J, which can optionally have one or morebatteries. A processor on the device, in some embodiments, is configuredto monitor a charge level of the device, such as a total amount ofbattery charge remaining. In some embodiments, charge-level is comparedto the maximum charge capacity of the battery. A processor on thedevice, in some embodiments, is further configured to monitorbattery-usage patterns of the device. Battery-usage patterns canoptionally include historical and contextual information about batteryusage and charge level, as will be explained in greater detail below.

At block 804, optionally, the battery usage patterns being monitoredinclude one or more of: battery amount used since activating a firstmode of operation, rate of battery usage since activating a first modeof operation, battery amount used since a past contextual event, rate ofbattery usage since the past contextual event, predicted time of thebattery exhaustion reaching a predetermined threshold level, andpredicted battery level by the time of a future contextual event.Accordingly, battery usage patterns include, in some embodiments,historical and contextual information about battery charge level,battery use rate, time of battery exhaustion, and how all of thesefactors relate to usage patterns and usage modes of the device.

In some embodiments, monitored battery-usage patterns include batteryamount used since activating a first mode of operation. In someembodiments, the device monitors a total amount of battery charge thathas been used since entering a mode of operation, for example byrecording an indication of the battery level at the time that the modeof operation is initiated, and comparing that initial amount of batterycharge with the amount of battery charge remaining at later times todetermine a charge level used since initiating the mode of operation.

In some embodiments, the first mode of operation is a high-power mode ofoperation. In some embodiments, the first mode of operation is anapplication. In some embodiments, the first mode of operation is a modein which a data interface, such as a Bluetooth, Wi-Fi (e.g., receivingWi-Fi data and/or acting as a Wi-Fi hotspot), or cellular data interfaceis enabled. In some embodiments, the first mode of operation is a modein which a data connection with a particular peripheral device (e.g., aperipheral display unit such as peripheral display unit 612 in FIG. 6Aor 6L) is active/connected. In some embodiments, the first mode ofoperation is a gaming mode, a media playback mode, a media recordingmode, a media streaming mode, or a navigation mode.

For example, in some embodiments, a device monitors an amount of batteryused since the activation of a mode of operation in which a CarPlaywireless connection is connected. Monitoring the total amount of batteryamount used since activating the mode of operation allows the device toprovide helpful contextual information to a user, such as “You have used50% of your battery since activating CarPlay,” or “You have used 75% ofyour remaining battery since activating CarPlay.” This information canhelp a user intuitively understand the battery-life implications of hisusage patterns and make informed decisions about how to best conservebattery life going forward.

In some embodiments, monitored battery-usage patterns include rate ofbattery usage since activating a first mode of operation. In someembodiments, the device monitors a rate of battery usage sinceactivating a mode of operation, for example by recording a charge levelat a time of initiation of a first mode of operation and recording thetime of initiation of the first mode of operation, and comparing therecorded charge level and recorded time to a current time and currentcharge level to determine a rate of battery usage since activation ofthe first mode of operation. In some other embodiments, the device canoptionally periodically record the charge level and the time at variouspoints during use of a particular mode of operation, so that it canmonitor changes in the rate of usage during a first mode of operationover time.

For example, in some embodiments, a device monitors a rate of batteryused since the activation of a mode of operation in which a CarPlaywireless connection is connected. Monitoring the rate of battery usagesince activating the mode of operation allows the device to providehelpful contextual information to a user, such as “You're using 50% ofyour battery per hour since activating CarPlay.” This information canhelp a user intuitively understand the battery-life implications of hisusage patterns and make informed decisions about how to best conservebattery life going forward.

In some embodiments, monitored battery-usage patterns include batteryamount used since a past contextual event. In some embodiments, batteryamount used since the time of a past contextual event is monitored in asimilar manner as described above with respect to battery amount usedsince the time of activation of a first mode of operation. In someembodiments, past contextual events include a time at which a user gotin a car, a time at which a user left home or work, or a time at which auser arrived at a certain location (as determined by GPS data or othercontextual information). In some embodiments, past contextual eventsinclude a time at which an event began (as determined by calendar data,internet data, or other contextual data).

For example, in some embodiments, a device monitors an amount of batteryused since getting in the car. Monitoring the total amount of batteryamount used since a past contextual event allows the device to providehelpful contextual information to a user, such as “You have used 50% ofyour battery since getting in the car,” or “You have used 75% of yourremaining battery since getting in the car.” This information can help auser intuitively understand the battery-life implications of his usagepatterns and make informed decisions about how to best conserve batterylife going forward.

In some embodiments, monitored battery-usage patterns include rate ofbattery usage since a past contextual event. In some embodiments, rateof battery usage since a past contextual event can optionally bemonitored in a similar manner as described above with respect to rate ofbattery usage since activating a first mode of operation.

For example, in some embodiments, a device monitors a rate of batteryused since a getting in the car. Monitoring the rate of battery usageused since a past contextual event allows the device to provide helpfulcontextual information to a user, such as “You're using 50% of yourbattery per hour since getting in the car.” This information can help auser intuitively understand the battery-life implications of his usagepatterns, and make informed decisions about how to best conserve batterylife going forward.

In some embodiments, battery-usage patterns comprise consideration ofhistorical and contextual data and predictions or projections calculatedin accordance therewith. That is, in some embodiments, the device usesthe current charge level and/or the current rate of battery usage inorder to make predictions about how quickly the battery will beexhausted. In some embodiments, the device can optionally furtherconsider the current mode of operation or recent contextual events inthe context of past instances of using the current mode of operation orpast instances of similar contextual events. For example, the device canoptionally calculate a running average of battery-usage rate during afirst mode of operation or during a first contextual scenario (e.g.,while driving) or at a certain location (e.g., while at home, work,etc.). The device can optionally consider this average rate of batteryuse in addition to the current charge level and/or the currentbattery-usage rate in order to make a prediction about the usage of thebattery. Predictions include, in some embodiments, a time of the batteryreaching a predetermined minimum threshold, and/or a predicted batterylevel at a predetermined future time.

In some embodiments, monitored battery-usage patterns include apredicted time of the battery reaching a predetermined threshold level.In some embodiments, the predetermined threshold level is a level atwhich the device will automatically power down (e.g., an “exhaustion”level). In some embodiments, the predetermined threshold level is alevel at which the device will enter a power-saving mode. In someembodiments, the predetermined threshold level is a percentage of thetotal charge capacity, such as 20% or 10%.

In some embodiments, monitored battery-usage patterns include apredicted battery level by the time of a future contextual event. Insome embodiments, future contextual events include the predicted endtime of a mode of operation, which can optionally be based on pastpatterns of usage of the mode (e.g., a historical average). In someembodiments, predicted end time of a mode of operation is determinedbased on or on cues from the mode (e.g., an application) itself, such asthe length of a document, the length of a media being played back, orthe number of levels in a game. In some embodiments, predicted end timeof a mode of operation is determined based on calendar data indicatingthe beginning or end of an event, internet data indicating the beginningor end of an event, or GPS data indicating a predicted arrival at adestination. In some embodiments, future contextual events include thepredicted arrival by a user at a destination. Time of arrival by a userat a destination can optionally be predicted, in some embodiments, byconsidering GPS data provided by the device, GPS data provided by aperipheral to which the device is connected, map data, internet datasuch as traffic data, and/or historical patterns of user conduct. Insome embodiments, future contextual events include the beginning or endof a calendar event.

For example, in some embodiments, a device predicts a time by which abattery will be exhausted if it continues to be used in media playbackmode, based on the current charge level of the battery, rate of batteryuse since beginning media playback mode, and a historical average ofbattery-usage rate in media playback mode. Then, the device compares thepredicted time of exhaustion to upcoming contextual events, anddetermines that the battery will likely be exhausted by media playbackmode before a user completes his drive home from work. The device canoptionally then provide a notification to the user indicating as much.This information can help a user intuitively understand the battery-lifeimplications of his usage patterns and make informed decisions about howto best conserve battery life going forward.

Blocks 806-810 occur while monitoring battery usage of the device, asexplained with reference to block 802.

At block 808, in accordance with a determination that a charge level ofthe device meets charge-level notification criteria, the device providesa charge-level alert that indicates a current charge level of the one ormore batteries. Charge-level notification criteria, in some embodiments,are predefined rules for triggering the provision of a charge-levelalert. In some embodiments, charge-level notification criteria includethat the charge level of the battery is below a predefined charge level,such as a predefined percentage of the total charge capacity of thebattery.

In some embodiments, the charge-level alert is provided by displaying analert on a display of the device, such as display 608 in FIG. 6J. Insome embodiments, the alert is provided by auditory or haptic output, orby sending a signal to another device or a connected peripheralinstructing the other device/peripheral to provide the alert. In someembodiments, the charge-level alert indicates a current charge level asa percentage of the total charge capacity of the battery. An exemplarycharge-level alert is alert 652 in FIG. 6J.

At block 810, in accordance with a determination that battery usage ofthe device meets battery-usage notification criteria different from thecharge-level notification criteria, the device provides a battery-usagealert that indicates a current battery usage pattern. Battery-usagenotification criteria, in some embodiments, are predefined rules fortriggering the provision of a battery-usage alert. As will be explainedin greater detail below, battery-usage notification criteria, in someembodiments, include a criterion that is met when a battery is predictedto be below a certain charge level by a certain future time.

In some embodiments, the battery-usage alert is provided by displayingan alert on a display of the device, such as display 608 in FIG. 6K. Insome embodiments, the alert is provided by auditory or haptic output, orby sending a signal to another device or a connected peripheralinstructing the other device/peripheral to provide the alert. In someembodiments, the battery-usage alert indicates a monitored battery-usagepattern. An exemplary battery-usage alert is alert 654 in FIG. 6K.

Attention is now directed to method 804-a in FIG. 8B. Method 804-a is anoptional process performed as part of block 804. That is, the steps inmethod 804-a primarily expand on the features regarding thebattery-usage patterns being monitored, as explained above withreference to block 804.

At block 812, optionally, the first mode of operation comprises ahigh-power mode of operation. In some embodiments, a high-power mode ofoperation is any application, hardware configuration, physical devicelocation, or other mode of operation that is predetermined or activelydetermined by the device to be a mode of operation in which batterycharge is likely to be used at an elevated rate. For example, high-powermodes of operation can optionally comprise application use, enablementof a data interface (such as a Bluetooth, Wi-Fi, or cellular datainterface), or connection with a peripheral device (e.g., a peripheraldisplay unit such as peripheral display unit 612 in FIG. 6A or 6L).

At block 814, optionally, the first mode of operation comprises awireless connection between the device and a peripheral display unit. Anexemplary peripheral display unit is the peripheral display unit of FIG.6A or 6L, and an exemplary wireless connection is any of the wirelessconnections described above with reference to FIGS. 7A-7H. Battery-usagealerts are particularly useful in this context because wirelessconnections between devices and peripheral display units, such as motorvehicles, are both energy-intensive and novel to users. Particularly,users may not be familiar with the battery implications of a Wi-Ficonnection, and battery usage alerts can therefore help users conservebattery.

At block 816, optionally, one or more of the charge-level alert (ofblock 808) and the battery-usage alert (of block 810) are providedthrough the peripheral display unit. In some embodiments, the devicesends data over the wireless connection to the peripheral display unitinstructing the peripheral display unit to provide the alert, such as bydisplaying it, providing an audio alert, or providing a haptic alert. Anexemplary provision of a battery-usage alert by both a device and aconnected peripheral display unit is shown by device 600 and peripheraldisplay unit 610 in FIG. 6L. Providing an alert on a peripheral displayunit, such as in a vehicle, can be particularly useful when users areprimarily interacting with the peripheral display unit rather than thedevice itself; as a user might not see or notice an alert provided onlyon the device, an alert provided through the peripheral display unit maybe more noticeable and useful.

At block 818, optionally, one or more of the charge-level alert (ofblock 808) and the battery-usage alert (of block 810) are providedthrough the peripheral display unit only when the device is not beingcharged. In some embodiments, even if the device would otherwise providean alert, the alert is not provided through the peripheral display unitif the device is being charged. For example, if the device is pluggedinto a car, wall outlet, computer, or other charging source, then thealert can optionally be provided only by the device itself, or not atall.

Attention is now directed to method 810-b in FIG. 8B. Method 810-b is anoptional process performed as part of block 810. That is, the steps inmethod 810-b primarily expand on the features regarding thebattery-usage patterns being monitored, as explained above withreference to block 810.

At block 820, optionally, battery-usage notification criteria comprise acriterion that is met when a charge level of the one or more batteriesof the device will be below a predefined charge level at a predicted endtime of usage of a first mode of operation. In some embodiments, asexplained above, the device relies on a combination of the currentcharge level, the rate of battery usage in the current mode ofoperation, historical rates of usage in the current mode of operation,and/or a prediction (e.g., based on historical usage patterns or presentcontextual indicators like media length or calendar-event length) ofwhen use of the first mode of operation will end; relying on some or allof these factors, in some embodiments, the device determines that thebattery is likely to fall below a predefined threshold before apredicted end time of usage of the first mode of operation. Inaccordance with such a determination, the device provides a batteryusage alert indicating its prediction. For example, an alert informs auser, “Your battery is going to be below 20% by the time this moviefinishes playing.”

At block 822, optionally, battery-usage notification criteria comprise acriterion that is met when a charge level of the one or more batteriesof the device will be below a predefined charge level at time of apredefined future event. In some embodiments, the predefined futureevent is based on calendar data (such as data entered in a user'spersonal calendar). For example, an alert informs a user, “Your batteryis going to be below 20% by the time the concert starts.”

Attention is now directed to blocks 824-828 in FIG. 8C. Blocks 824-828are part of process 800 for monitoring battery-usage patterns andproviding alerts in accordance with some embodiments. Blocks 824-828optionally occur following blocks 806-810 in FIG. 8A.

At block 824, optionally, the device determines a destination for thedevice, wherein the battery-usage notification criteria comprise acriterion that is met when a charge level of the one or more batteriesof the device will be below a predefined charge level at a predictedtime at which the device will arrive at the destination. In accordancewith some embodiments, the device considers the current position of thedevice (as determined by GPS data), a determined destination of thedevice, map data, traffic data (e.g., live traffic data from theinternet), and any or all of the charge-level data and battery-usagepatterns discussed above; in accordance with considering some or all ofthis data, the device predicts a time at which the user is expected toreach his destination and a charge level at which the battery ispredicted to be at that future time. In some embodiments, abattery-usage notification criterion considers whether that predictedcharge level is below a predetermined threshold; if the battery will bebelow such a threshold at the time a user is predicted to arrive at thedestination, then, in some embodiments, the device provides abattery-usage alert indicating its prediction for the user. An exemplarybattery usage alert provided in accordance with such a battery-usagenotification criterion is alert 656 in FIG. 6L.

At block 826, optionally, the destination is a destination explicitlyidentified by the user. In some embodiments, the destination is adestination explicitly provided in a GPS application, a navigationapplication, a search application, a calendar application, or by someother explicit entry means.

At block 828, optionally, the destination is a destination predictedbased on user behavior. In some embodiments, the device predicts a userdestination based on the user's past behavior, such as repeatedlydriving to and from the same home and place of work on every weekday, orrepeatedly driving to the same restaurant on every Monday evening.

Attention is now directed to blocks 830-834 in FIG. 8D. Blocks 830-834are part of process 800 for monitoring battery-usage patterns andproviding alerts in accordance with some embodiments. Blocks 830-834optionally occur following blocks 806-810 in FIG. 8A.

At block 830, optionally, in accordance with providing one or more ofthe charge-level alert and the battery-usage alert, the device providesan option to deactivate the first mode of operation. In someembodiments, in accordance with the same criteria that trigger displayof a charge-level alert or a battery-usage alert, the device providesthe user with an option to deactivate a first mode of operation. In someembodiments, the device can optionally indicate that the mode is ahigh-power mode of operation, or that the mode should be disabled toconserve battery power. In some embodiments, the device presents theoption to deactivate the mode as a default option, such as by displayingan affordance to deactivate the mode of operation in bolded font, or ina different color. An exemplary option to deactivate a first mode ofoperation is CarPlay deactivation prompt 658 in FIG. 6M, which promptsthe user to deactivate a CarPlay wireless connection mode.

At block 832, optionally, the device detects selection of the option. Anexemplary selection of the option is user input 660-a or user input660-b in FIG. 6M, which is a touch or tap input detected on anaffordance displayed as part of CarPlay deactivation prompt 658 oneither or both of display 608 or display 618.

At block 834, optionally, in accordance with detecting selection of theoption, the device deactivates the first mode of operation. In someembodiments, in response to detecting user input 660-a or 660-b, thedevice disconnects a wireless CarPlay connection, includingdisconnecting the Wi-Fi data connection associated therewith.

Attention is now directed to block 836 in FIG. 8D. Block 836 is part ofprocess 800 for monitoring battery-usage patterns and providing alertsin accordance with some embodiments. Block 836 optionally occursfollowing blocks 806-810 in FIG. 8A.

At block 836, optionally, in accordance with a determination that acharge level of the device meets charge-level notification criteria, thedevice automatically deactivates a first mode of operation. In someembodiments, the device automatically discontinues a high-power mode ofoperation in accordance with determining that the charge-level of thedevice is below a predetermined threshold. In some embodiments, thedevice automatically disconnects a wireless connection, such as any ofthose described above with reference to FIGS. 7A-7H. The disconnectioncan optionally occur without any user intervention or approval.

It should be understood that the particular order in which theoperations in FIGS. 8A-8D have been described is merely exemplary and isnot intended to indicate that the described order is the only order inwhich the operations could be performed. One of ordinary skill in theart would recognize various ways to reorder the operations describedherein. Additionally, it should be noted that details of other processesdescribed herein with respect to other methods described herein (e.g.,methods 700, 900) are also applicable in an analogous manner to 800described above with respect to FIGS. 8A-8D. For example, the devices,peripheral display units, relationships, connections, and userinterfaces described above with reference to 800 optionally have one ormore of the characteristics of the devices, peripheral display units,relationships, connections, and user interfaces described herein withreference to other methods described herein (e.g., methods 700, 900).

FIGS. 9A-9E are flow diagrams illustrating exemplary process 900 forconfiguring a user interface of a peripheral display unit in accordancewith some embodiments, such as those described above with reference toFIGS. 6N-6W. Process 900 is carried out by device 500 (FIG. 5), whichcan optionally be electronic device 100 or 300 (FIGS. 1A and 3A), insome embodiments.

Attention is now directed to FIG. 9A, which is a flow diagramillustrating an exemplary process for configuring a user interface of aperipheral display unit. In accordance with the embodiments describedherein, users can conveniently and efficiently configure the userinterface of peripheral display units associated with a device. Thisconfiguration can be done remotely (away from the peripheral displayunit) while the device is not actively connected to the peripheraldisplay unit. Remote configuration is convenient to the user because itprovides flexibility as to the time and place at which configuration canbe done. Additionally, configuration via an associated device may bemore convenient if the device has a superior input interface, such aswhen a mobile phone has a modern capacitive touch screen and aperipheral display unit has a cumbersome input interface. Furthermore,in embodiments in which the peripheral display unit is provided in avehicle, allowing configuration of the interface improves driver safetyby making the user interface more intuitive, customizable, and quicklyaccessible. Furthermore, allowing remote configuration lessens the needfor a user to attempt configuration operations while in the car,improving driver safety.

At block 902, optionally, the device establishes a relationship betweenthe device and the peripheral display unit. In some embodiments,establishing such a relationship can optionally include establishing arelationship according to any of the methods described above withreference to FIGS. 7A-7H.

At block 904, the device displays on the display of the device aconfiguration interface, wherein the configuration interface comprises arepresentation of a user interface of a peripheral display unit. In someembodiments, the representation of the peripheral interface is a windowin an interface on the device, the window depicting a likeness of theperipheral interface. An exemplary configuration interface is interface664 in FIG. 6N. Interface 664 includes window 668, which is an exemplaryrepresentation of a user interface of a peripheral display unit. Window668 represents peripheral interface 662, which is an exemplary userinterface of a peripheral display unit.

At block 906, the device detects a request to configure the userinterface of the peripheral display unit. In some embodiments, therequest is a touch input detected by a touch-sensitive surface at alocation corresponding to the configuration interface and/or therepresentation of the peripheral user interface displayed therein. Insome embodiments, the request to configure the user interface representsa request to add a new user interface object to the peripheralinterface, remove a user interface object from the peripheral interface,or rearrange a user interface object on the peripheral interface, aswill be described in greater detail below.

At block 910, optionally, displaying on the display of the device anupdated representation of the user interface occurs while a dataconnection corresponding to a relationship between the device and theperipheral display unit is not active. In some embodiments, theconfiguration interface is accessible and fully functional when thedevice is not actively connected with the peripheral display unit.Accordingly, while no connection is active, a user in some embodimentscan optionally access the configuration interface and executeinstructions to configure the user interface of the peripheral device.In some embodiments, the configuration interface will display anindication to the user, such as a reconfigured representation of theperipheral interface, that a configuration has occurred, despite theperipheral display unit not being connected to the device. In someembodiments, the device stores data representing any configuration thatthe user has executed.

At block 912, after detecting the request to configure the userinterface of the peripheral display unit, the device transmitsinstructions to the peripheral display unit to display the userinterface generated in accordance with the detected request. In someembodiments, after detecting the request to configure the peripheralinterface via the configuration interface, the device transmits data tothe peripheral display unit representing an instruction to execute theconfiguration. In embodiments where the device and peripheral displayunit are connected via an active connection when the instruction isdetected, the transmission can optionally occur immediately, or verysoon, after the instruction.

At block 914, optionally, transmitting instructions to the peripheraldisplay unit occurs when the data connection corresponding to therelationship between the device and the peripheral display unit isestablished. In some embodiments where the device and peripheral displayunit are not connected via an active connection when the instruction isdetected, the transmission can optionally occur at a later time in thefuture, such as upon the next reestablishment of a connection betweenthe device and the peripheral display unit. The transmission ofinstructions can optionally occur over the data connection associatedwith any of the relationships discussed above. For example, thetransmission of instructions, in some embodiments, occurs over the Wi-Fidata connection of a wireless CarPlay connection. In some embodiments,the user may be unaware that the transmission was delayed from the timeof the instruction, because the user will not have interacted with theperipheral at any time before the connection automatically reestablishesitself in accordance with the methods explained above with reference toFIGS. 7A-7H.

In FIG. 9B, at block 916, optionally, the device stores the instructionsin a memory of the device at least until the data connectioncorresponding to the relationship between the device and the peripheraldisplay is established. In some embodiments, when the configurationinstruction is received at a time when a connection between the deviceand the peripheral display unit is not active, the device stores data ona local memory representing the instruction to configure the peripheraldisplay unit. In some embodiments, the configuration instruction is thentransmitted, as explained above with reference to block 914, when theconnection is next reestablished. In some embodiments, the configurationinstructions are maintained in a memory on the device both before andafter the relationship is next reestablished, as data corresponding tothe user interface of the peripheral display unit is permanentlymaintained on the device and served to the peripheral display unitwhenever the connection is active.

Attention is now directed to methods 910-a, 910-b, and 910-c in FIG. 9B.Methods 910-a, 910-b, and 910-c are methods, in that they are optionalprocesses performed as part of block 910. That is, the steps in methods910-a, 910-b, and 910-c primarily expand on the features regarding theconfiguration instructions transmitted from the device to the peripheraldisplay unit, as explained above with reference to block 910.

As will be explained below, methods 910-a, 910-b, and 910-c representtechniques for adding, removing, and repositioning user interfaceobjects on the peripheral user interface, respectively.

At block 918, optionally, the configuration user interface comprises aset of one or more representations of user interface objects availableto be added to the user interface of the peripheral display unit, theinstructions comprise instructions for adding to the user interface auser interface object corresponding to one of the one or morerepresentations of user interface objects, and the request to configurethe user interface comprises a user input corresponding to a location ofone of the one or more representations of user interface objectsavailable to be added to the user interface.

An exemplary set of one or more representations of user interfaceobjects available to be added to the user interface of the peripheraldisplay unit is the set of icons in tray 670 in FIG. 6N. In someembodiments, representations of available user interface objectsrepresent icons that can optionally be added to the peripheralinterface, such as icons corresponding to new applications or newfunctions that have been downloaded by a user of the device or a user ofthe peripheral display unit. In some embodiments, the representationshave been pushed to the configuration interface by a back-end operatorof the configuration interface, such as a developer associated with thedevice or the peripheral display unit.

In some embodiments, a new user interface object is added to aperipheral interface in accordance with an input that drags arepresentation of a new object onto the representation of the peripheralinterface. FIGS. 6N-6P represent an example of an instruction detectedat a device user interface to add an icon to the peripheral userinterface. In FIG. 6N, user input 674-a is detected. User input 674 is aswipe input that continues as user input 674-2 in FIG. 6O, where itlifts off In accordance with the input in FIGS. 6N and 6O, exemplaryrepresentation 672-a is displayed as moving from tray 670 to window 668.In embodiments in which the device and the peripheral display unit arenot connected at the time, the instruction to update the peripheraldisplay unit is not yet transmitted, as illustrated in FIG. 6O, in whichperipheral interface 662 is not yet updated. FIG. 6P depicts theinstruction to add the new user interface object being transmitted tothe peripheral display unit and implemented by the peripheral displayunit, as new user interface object 676 is displayed.

At block 920, optionally, the instructions comprise instructions forremoving a user interface object from the user interface. In someembodiments, a user interface object is removed from a peripheraldisplay interface in accordance with a gesture that selects a deletionaffordance associated with a representation of a user interface icon ona representation of a peripheral user interface.

FIGS. 6Q-6S represent an example of an instruction detected at a deviceuser interface to remove an icon from the peripheral user interface. InFIG. 6Q, user input 680 is detected. User input 674 is a tap or pressinput detected at a location corresponding to deletion affordance 678,which corresponds to representation 672-a. In accordance with input 680,exemplary representation 672-a ceases to be displayed in window 668 andis displayed in tray 670, as shown in FIG. 6R. In embodiments in whichthe device and the peripheral display unit are not connected at thetime, the instruction to update the peripheral display unit is not yettransmitted, as illustrated in FIG. 6R, in which peripheral interface662 is not yet updated. FIG. 6S depicts the instruction to remove theuser interface object being transmitted to the peripheral display unitand implemented by the peripheral display unit, as user interface object676 is no longer displayed.

At block 922, optionally, the instructions comprise instructions forrepositioning of a user interface object on the user interface. In someembodiments, a user interface object is repositioned on a peripheraldisplay interface in accordance with an input that drags arepresentation of a user interface object from one location on arepresentation of the user interface to another location on therepresentation of the user interface.

FIGS. 6T-6V represent an example of an instruction detected at a deviceuser interface to reposition an icon on the peripheral user interface.In FIG. 6T, user input 682-1 is detected. User input 682 is a swipeinput that continues as user input 682-2 in FIG. 6U, where it lifts off.In accordance with the input in FIGS. 6T and 6U, exemplaryrepresentation 672-a is displayed as moving from a first position inwindow 668 to a second position in window 668. In embodiments in whichthe device and the peripheral display unit are not connected at thetime, the instruction to update the peripheral display unit is not yettransmitted, as illustrated in FIG. 6U, in which peripheral interface662 is not yet updated. FIG. 6V depicts the instruction to add the newuser interface object being transmitted to the peripheral display unitand implemented by the peripheral display unit, as user interface object676 is displayed in its new position.

Attention is now directed to FIGS. 9C-9E, which are flow diagramsillustrating an exemplary process for configuring a user interface of aperipheral display unit. The steps represented in FIGS. 9C-9E occurfollowing blocks 912 and 914 discussed above.

At block 924, the configuration interface further comprises arepresentation of a second user interface of a second peripheral displayunit. In some embodiments, a second representation of a secondperipheral display unit is displayed at the same time as the firstrepresentation of the first user interface. In some embodiments, theconfiguration user interface selectively displays differentrepresentations of the user interfaces of different peripheral displayunits one at a time. An exemplary configuration user interfacedisplaying a second representation of a second peripheral display unitis interface 664 in FIG. 6W, which is displaying window 686. Window 686is a representation of user interface 684, which is the user interfaceof peripheral display unit 610-b. Displaying representations of morethan one user interface allows a user to use the configuration interfaceto configure user interfaces for multiple peripheral display units froma single device. For example, if a user has multiple CarPlay-enabledcars with which he has established relationships with his device, thenthe user can use a single device to configure the user interface of bothcars. Conveniently, this can be done from a single device, and at anytime while actively connected to either car or to neither of the cars.

At blocks 926-938, optionally, steps are carried out in which the devicedetects an input, via the configuration interface, to configure thesecond peripheral interface. In response to the request, which in someembodiments occurs while the device and the second peripheral are notactively connected via a connection, the device displays an updatedrepresentation of the user interface in accordance with theconfiguration request. Then, after detecting the request, and in someembodiments when the device reestablishes a connection with the secondperipheral, the device transmits instructions to the second peripheralto display the second peripheral interface in accordance with the secondconfiguration instruction. In embodiments in which the device and theperipheral are not connected at the time that the configurationinstruction is detected, the device stores the instruction locally atleast until it is transmitted to the second peripheral. All of thesesteps can optionally be carried out, in some embodiments, in all of thesame manners as described above with reference to the first peripheraluser interface and blocks 902-922.

At block 926, optionally, the device detects a second request toconfigure the second user interface.

At block 928, optionally, in response to detecting the second request toconfigure the second user interface, the device displays on the displayof the device a second updated representation of the second userinterface, wherein the second updated representation is generated inaccordance with the second detected request.

At block 930, optionally, displaying on the display of the device thesecond updated representation of the second user interface occurs whilea data connection corresponding to the relationship between the deviceand the second peripheral display unit is not active.

At block 932, optionally, after detecting the second request toconfigure the second user interface, the device transmits instructionsto the second peripheral display unit to display the second userinterface in accordance with the detected request.

At block 934, optionally, transmitting second instructions to the secondperipheral display unit occurs when the data connection corresponding tothe relationship between the device and the second peripheral displayunit is established.

At block 936, optionally, the device stores the second instructions in amemory of the device at least until the data connection corresponding tothe relationship between the device and the second peripheral displayunit is established.

At block 938, optionally, the first instruction and the secondinstruction are simultaneously stored at the device.

At block 940, the first instruction and the second instruction aresimultaneously stored at the device. In some embodiments, the devicedetects a first instruction for configuring a user interface of a firstdevice, and detects a second instruction for configuring the userinterface of a second device. In some embodiments, the device receivesboth instructions while disconnected from the peripheral display unitsassociated with both peripheral interfaces. In some embodiments, thedevice stores both instructions on a local memory at the same time, atleast until one or both is transmitted to the associated peripheraldisplay unit. Each instruction can optionally be indexed by a fileidentifier associated with the corresponding peripheral display unit.

Attention is now directed to method 924-f in FIG. 9E. Method 924-f is anoptional process performed as part of block 924. That is, the steps inmethod 924-f primarily expand on the features regarding the second userinterface of a second peripheral display unit, as explained above withreference to block 924.

At block 942, optionally, the second user interface comprises one ormore user interface objects different from the first user interface. Insome embodiments, the user interface objects have a different appearanceand/or a different function. In some embodiments, the user interfaceobjects on the second user interface correspond to applications that areunique to the manufacturer or provider of the second peripheral displayunit. An example of the user interface objects on different peripheralinterfaces being different is depicted by the comparison of userinterface objects 663 on first peripheral interface 662 in FIG. 6V anduser interface objects 668 in interface 684 in FIG. 6W.

At block 944, optionally, the second user interface comprises anarrangement of user interface objects different from the first userinterface. An example of the user interface objects on differentperipheral interfaces being in different arrangements is depicted by thecomparison of user interface objects 663 on first peripheral interface662 in FIG. 6V and user interface objects 668 in interface 684 in FIG.6W.

At block 946, optionally, the second peripheral display unit has one ormore of a screen size different from the first peripheral display unit,a screen orientation different from the first peripheral display unit,and a screen resolution different from the first peripheral displayunit. An example of such differences is depicted by the comparison ofdevice 610 having screen 618 in FIG. 6V and device 610-b having screen618-b in FIG. 6W. The screen size, screen resolution, and screenorientation of display 618 and 618-b are all different.

At block 948, optionally, the configuration user interface comprises asecond set of representations of one or more user interface objectsavailable to be added to the second user interface, the secondinstructions comprise instructions for adding to the user interface auser interface object corresponding to one of the one or morerepresentations of user interface objects from the second set, thesecond request to configure the second user interface comprises a seconduser input corresponding to a location of the one of the one or morerepresentations of user interface objects from the second set, and thefirst set and the second set comprise representations of different userinterface objects. In some embodiments, a new user interface object isadded to the second peripheral interface in accordance with an inputthat drags a representation of a new object onto the representation ofthe second peripheral interface. In some embodiments, this process canoptionally be carried out in all of the same manners as described abovewith reference to adding new user interface objects to the first userinterface, as in block 918. However, in embodiments in which thereconfiguration interface displays different representations ofdifferent peripheral interfaces, the configuration interface canoptionally also display a different set of representations of availablenew user interface icons. In some embodiments, each user interfaceavailable for configuration has a corresponding set of representationsof available new user interface objects. The available user interfaceicons can optionally be unique to each peripheral user interface inaccordance with different user interface objects made available, in someembodiments, by developers or manufacturers associated with specificperipheral display units. For example, the manufacturers of differentcars having built-in peripheral display units can optionally makeavailable different applications that are specifically available to thatmanufacturer's cars.

It should be understood that the particular order in which theoperations in FIGS. 9A-9E have been described is merely exemplary and isnot intended to indicate that the described order is the only order inwhich the operations could be performed. One of ordinary skill in theart would recognize various ways to reorder the operations describedherein. Additionally, it should be noted that details of other processesdescribed herein with respect to other methods described herein (e.g.,methods 700, 800) are also applicable in an analogous manner to 900described above with respect to FIGS. 9A-9E. For example, the devices,peripheral display units, relationships, connections, and userinterfaces described above with reference to 900 optionally have one ormore of the characteristics of the devices, peripheral display units,relationships, connections, and user interfaces described herein withreference to other methods described herein (e.g., methods 700, 800).

In accordance with some embodiments, FIG. 10 shows a functional blockdiagram of an electronic device 1000 configured in accordance with theprinciples of the various described embodiments. The functional blocksof the device are, optionally, implemented by hardware, software, or acombination of hardware and software to carry out the principles of thevarious described embodiments. It is understood by persons of skill inthe art that the functional blocks described in FIG. 10 are, optionally,combined or separated into sub-blocks to implement the principles of thevarious described embodiments. Therefore, the description hereinoptionally supports any possible combination or separation or furtherdefinition of the functional blocks described herein.

As shown in FIG. 10, electronic device 1000 includes display unit 1002configured to display a user interface, input interface unit 1004configured to receive input, wireless communication unit 1006 configuredto send and/or receive data over a wireless data connection, wirelesscommunication unit 1008 configured to send and/or receive data over awireless data connection, wired communication unit 1010 configured tosend and/or receive data over a wired data connection, and processingunit 1014 coupled to said units. In some embodiments, processing unit1014 includes receiving unit 1016, establishing unit 1018, detectingunit 1020, receiving enabling unit 1022, displaying enabling unit 1024,providing enabling unit 1026, transferring enabling unit 1028,communicating enabling unit 1030, storing unit 1032, and activating unit1034.

Processing unit 1014 is configured to receive (e.g., with the receivingunit 1016) a request from a user to authorize a relationship thatcorresponds to a connection between the device and the peripheraldisplay unit over a first data connection with the peripheral displayunit. Processing unit 1014 is further configured to, in response toreceiving the request to authorize a relationship, establish (e.g., withthe establishing unit 1018) a relationship with the peripheral displayunit, wherein establishing the relationship includes enabling receiving(e.g., with the enabling receiving unit 1020) authentication informationfrom the peripheral display unit via a second data connection that isdifferent from the first data connection. Processing unit 1014 isfurther configured to, while a connection between the device and theperipheral display unit over the first data connection is not active,detect (e.g., with the detecting unit 1020) that the peripheral displayunit is available for establishment of a connection, and, in response todetecting that the peripheral display unit is available forestablishment of a connection, establish (e.g. with the establishingunit 1018) a connection between the device and the peripheral displayunit, wherein the connection is a connection over the first dataconnection, and wherein establishing the connection between the deviceand the peripheral display unit comprises enabling providing (e.g., withthe providing enabling unit) the authentication information to theperipheral display unit to establish the connection.

In some embodiments, the second data connection is a wired dataconnection.

In some embodiments, processing unit 1014 is further configured to,while a relationship has not been established with the peripheraldisplay unit, enable receiving (e.g., via receiving enabling unit 1020)a communication from the peripheral display unit over the wired dataconnection between the device and the peripheral display unit. In someembodiments, processing unit 1014 is further configured to, in responseto receiving the communication, enable displaying (e.g., with displayingenabling unit 1022) a first user interface prompting the user toauthorize a relationship with the peripheral display unit.

In some embodiments, displaying (e.g., via display unit 1002) the firstuser interface prompting the user to authorize a relationship comprisesdisplaying a first option corresponding to authorizing a wirelessrelationship between the device and the peripheral display unit, asecond option corresponding to authorizing a wired relationship betweenthe device and the peripheral display unit, and a third optioncorresponding to authorizing a charging-only connection between thedevice and the peripheral display unit.

In some embodiments, the request from the user to establish (e.g., viaestablishing unit 1018) a relationship with the peripheral display unitcomprises selection of the first option.

In some embodiments, the authentication information comprisescredentials for a wireless data connection distinct from the first dataconnection.

In some embodiments, the second data connection is a wireless dataconnection.

In some embodiments, processing unit 1014 is further configured to,while a relationship has not been established (e.g., via establishingunit 1018) with the peripheral display unit: detect (e.g., withdetecting unit 1020), via one or more of the first and second dataconnections, that the peripheral display unit is available for theestablishment of a relationship. In some embodiments, processing unit1014 is further configured to, in response to detecting (e.g., viadetecting unit 1020) that the peripheral display unit is available forthe establishment of a relationship, enable displaying (e.g., viadisplaying enabling unit 1022) a fourth option for authorizing arelationship with the peripheral display unit, wherein the request froma user to establish a relationship comprises selection of the fourthoption.

In some embodiments, processing unit 1014 is further configured todetect (e.g., with detecting unit 1020) selection of the fourth option.In some embodiments, processing unit 1014 is further configured toenable displaying (e.g., with displaying enabling unit 1022) a seconduser interface prompting the user to authorize a relationship with theperipheral display unit, wherein the second user interface comprises: afifth option corresponding to authorizing a wireless relationshipbetween the device and the peripheral display unit, and a sixth optioncorresponding to authorizing a wired relationship between the device andthe peripheral display unit.

In some embodiments, the request from the user to establish (e.g., withestablishing unit 1018) a relationship with the peripheral display unitcomprises selection of the fifth option.

In some embodiments, the fourth option for authorizing a relationship isan item in a list of Bluetooth-capable devices.

In some embodiments, the fourth option for authorizing a relationship isan item in a list of vehicles with peripheral display units.

In some embodiments, establishing (e.g., with establishing unit 1018) arelationship with the peripheral display unit includes enablingdisplaying (e.g., with displaying enabling unit 1022) on the display ofthe device a third user interface prompting the user to entercredentials for the wireless data connection.

In some embodiments, establishing (e.g., with establishing unit 1018) arelationship comprises enabling displaying (e.g., with displayingenabling unit 1022) a fourth user interface providing the user with anoption to establish a relationship with the peripheral display unitwithout syncing device contacts with the peripheral display unit.

In some embodiments, processing unit 1014 is further configured to,while a connection between the device and the peripheral display unitover the first data connection is active, enable transferring (e.g.,with transferring enabling unit 1026) content over the first dataconnection.

In some embodiments, detecting (e.g., with detecting unit 1020) that theperipheral display unit is available for establishment of a connectionoccurs while the device is in a locked state.

In some embodiments, establishing (e.g., with establishing unit 1018) aconnection between the device and the peripheral display unit occurswhile the device remains in a locked state.

In some embodiments, establishing (e.g., with establishing unit 1018)the connection between the device and the peripheral display unitfurther comprises: enabling communicating (e.g., with communicatingenabling unit 1028) with the peripheral display unit via a wireless dataconnection distinct from the first data connection, wherein a messagecommunicated over the wireless data connection comprises an explicitinstruction to establish a connection via the first data connection.

In some embodiments, the instruction to establish (e.g., withestablishing unit 1018) a connection via the first data connection istransferred from the peripheral display unit to the device.

In some embodiments, establishing (e.g., with establishing unit 1018) arelationship with the peripheral display unit comprises storing (e.g.,with storing unit 1030) an indication of whether to establish a wirelessconnection, a wired connection, or a charge-only connection.

In some embodiments, the peripheral display unit has a discoverablestate and a non-discoverable state, and the peripheral display unitcomprises a hardware button configured to cause the peripheral displayunit to enter the discoverable state.

In some embodiments, processing unit 1014 is further configured to thehardware button is further configured to activate a native function ofthe peripheral display unit unrelated to making the unit discoverable.

In some embodiments, the hardware button is further configured toactivate (e.g., with activating unit 1032) a function of the devicedistinct from the native function of the peripheral display andunrelated to making the peripheral display unit discoverable.

In some embodiments, a short press of the hardware button activates anative voice recognition function of the peripheral display unit, and along press of the hardware button causes the peripheral display unit toenter the discoverable state.

In some embodiments, processing unit 1014 is further configured to,while a connection between the device and the peripheral display unitover the first data connection is active, detect (e.g., with detectingunit 1020) a long press of the hardware button. In some embodiments,processing unit 1014 is further configured to, in response to detectingthe long press of the hardware button, activate (e.g., with activatingunit 1032) the function of the device.

In accordance with some embodiments, FIG. 11 shows a functional blockdiagram of an electronic device 1100 configured in accordance with theprinciples of the various described embodiments. The functional blocksof the device are, optionally, implemented by hardware, software, or acombination of hardware and software to carry out the principles of thevarious described embodiments. It is understood by persons of skill inthe art that the functional blocks described in FIG. 11 are, optionally,combined or separated into sub-blocks to implement the principles of thevarious described embodiments. Therefore, the description hereinoptionally supports any possible combination or separation or furtherdefinition of the functional blocks described herein.

As shown in FIG. 11, electronic device 1100 includes display unit 1102configured to display a user interface, input interface unit 1104configured to receive input, wireless communication unit 1106 configuredto send and/or receive data over a wireless data connection, wirelesscommunication unit 1108 configured to send and/or receive data over awireless data connection, wired communication unit 1110 configured tosend and/or receive data over a wired data connection, and processingunit 1114 coupled to said units. In some embodiments, processing unit1114 includes monitoring unit 1116, providing enabling unit 1118,determining unit 1120, detecting unit 1122, and deactivating unit 1124.

Processing unit 1114 is configured to monitor (e.g. with monitoring unit1116) battery usage of the device, wherein monitoring (e.g., withmonitoring unit 1116) battery usage of the device includes monitoring(e.g., with monitoring unit 1116) a charge level of one or morebatteries of the device and monitoring (e.g., with monitoring unit 1116)battery usage patterns of the one or more batteries of the device.Processing unit 1114 is further configured to, while monitoring (e.g.,with monitoring unit 1116) battery usage of the device, in accordancewith a determination that a charge level of the device meetscharge-level notification criteria, enable providing (e.g. withproviding enabling unit 1118) a charge-level alert that indicates acurrent charge level of the one or more batteries; and, in accordancewith a determination that battery usage of the device meetsbattery-usage notification criteria different from the charge-levelnotification criteria, enable providing (e.g., with providing enablingunit 1118) a battery-usage alert that indicates a current battery usagepattern.

In some embodiments, the battery usage patterns being monitored includeone or more of: battery amount used since activating a first mode ofoperation, rate of battery usage since activating a first mode ofoperation, battery amount used since a past contextual event, rate ofbattery usage since a past contextual event, predicted time of thebattery reaching a predetermined threshold level, and predicted batterylevel by the time of a future contextual event.

In some embodiments, the first mode of operation comprises a high-powermode of operation.

In some embodiments, the first mode of operation comprises a wirelessconnection between the device and a peripheral display unit.

In some embodiments, one or more of the charge-level alert and thebattery-usage alert are provided (e.g., by providing enabling unit 1118)through the peripheral display unit.

In some embodiments, one or more of the charge-level alert and thebattery-usage alert are provided (e.g., by providing enabling unit 1118)through the peripheral display unit only when the device is not beingcharged.

In some embodiments, the battery-usage notification criteria comprise acriterion that is met when a charge level of the one or more batteriesof the device will be below a predefined charge level at a predicted endtime of usage of a first mode of operation.

In some embodiments, the battery-usage notification criteria comprise acriterion that is met when a charge level of the one or more batteriesof the device will be below a predefined charge level at time of apredefined future event.

In some embodiments, processing unit 1117 is further configured to,determine (e.g., with determining unit 1120) a destination for thedevice, wherein the battery-usage notification criteria comprise acriterion that is met when a charge level of the one or more batteriesof the device will be below a predefined charge level at a predictedtime at which the device will arrive at the destination.

In some embodiments, the destination is a destination explicitlyidentified by the user.

In some embodiments, the destination is a destination predicted based onuser behavior.

In some embodiments, the processing 1114 unit is further configured to,in accordance with providing one or more of the charge-level alert andthe battery-usage alert, enable providing (e.g., with providing enablingunit 1120) an option to deactivate a first mode of operation. In someembodiments, the processing 1114 unit is further configured to detect(e.g., with detecting unit 1122) selection of the option, and, inaccordance with detecting (e.g., with detecting unit 1122) selection ofthe option, deactivate (e.g., with deactivating unit 1124) the firstmode of operation.

In some embodiments, the processing 1114 unit is further configured to,in accordance with a determination that a charge level of the devicemeets charge-level notification criteria, automatically deactivate(e.g., with deactivating unit 1124) a first mode of operation.

In accordance with some embodiments, FIG. 12 shows a functional blockdiagram of an electronic device 1200 configured in accordance with theprinciples of the various described embodiments. The functional blocksof the device are, optionally, implemented by hardware, software, or acombination of hardware and software to carry out the principles of thevarious described embodiments. It is understood by persons of skill inthe art that the functional blocks described in FIG. 12 are, optionally,combined or separated into sub-blocks to implement the principles of thevarious described embodiments. Therefore, the description hereinoptionally supports any possible combination or separation or furtherdefinition of the functional blocks described herein.

As shown in FIG. 12, electronic device 1200 includes display unit 1202configured to display a user interface, input interface unit 1204configured to receive input, wireless communication unit 1206 configuredto send and/or receive data over a wireless data connection with aperipheral display unit, wireless communication unit 1208 configured tosend and/or receive data over a wireless data connection with aperipheral display unit, wired communication unit 1210 configured tosend and/or receive data over a wired data connection with a peripheraldisplay unit, and a processing unit 1214 coupled to said units. In someembodiments, the processing unit 1214 includes display enabling unit1216, detecting unit 1218, transmitting enabling unit 1220, establishingunit 1222, and storing unit 1224.

Processing unit 1214 is configured to enable displaying (e.g., withdisplay enabling unit 1216) on the display unit of the device aconfiguration interface, wherein the configuration interface comprises arepresentation of a user interface of a peripheral display unit.Processing unit 1214 is further configured to enable detecting (e.g.,with detecting unit 1218) a request to configure the user interface ofthe peripheral display unit. Processing unit 1214 is further configuredto enable, in response to detecting (e.g. with detecting unit 1218) therequest to configure the user interface, enable displaying (e.g., withdisplay enabling unit 1216) on the display unit of the device an updatedrepresentation of the user interface, wherein the updated representationis generated in accordance with the detected request. Processing unit1214 is configured to enable, after detecting (e.g., with detecting unit1218) the request to configure the user interface of the peripheraldisplay unit, transmitting (e.g., with transmitting enabling unit 1220)instructions to the peripheral display unit to display the userinterface generated in accordance with the detected request.

In some embodiments, processing unit 1214 is further configured to,before enabling displaying (e.g., with display enabling unit 1216) theconfiguration interface, establish (e.g., with establishing unit 122) arelationship between the device and the peripheral display unit.

In some embodiments, displaying on the display unit of the device anupdated representation of the user interface occurs while a dataconnection corresponding to a relationship between the device and theperipheral display unit is not active, and transmitting instructions tothe peripheral display unit occurs when the data connectioncorresponding to the relationship between the device and the peripheraldisplay unit is established (e.g., with establishing unit 1222).

In some embodiments, processing unit 1214 is further configured to store(e.g., with storing unit 1224) the instructions in a memory of thedevice at least until the data connection corresponding to therelationship between the device and the peripheral display isestablished (e.g., with establishing unit 1222).

In some embodiments, the configuration user interface comprises a set ofone or more representations of user interface objects available to beadded to the user interface of the peripheral display unit, theinstructions comprise instructions for adding to the user interface auser interface object corresponding to one of the one or morerepresentations of user interface objects, and the request to configurethe user interface comprises user input corresponding to a location ofone of the one or more representations of user interface objectsavailable to be added to the user interface.

In some embodiments, the instructions comprise instructions for removinga user interface object from the user interface.

In some embodiments, the instructions comprise instructions forrepositioning of a user interface object on the user interface.

In some embodiments, the configuration interface further comprises arepresentation of a second user interface of a second peripheral displayunit, and the processing unit is further configured to: detect (e.g.,with detecting unit 1218) a second request to configure the second userinterface; in response to detecting (e.g., with detecting unit 1218) thesecond request to configure the second user interface, enable displaying(e.g., with display enabling unit 1216) on the display unit of thedevice a second updated representation of the second user interface,wherein the second updated representation is generated in accordancewith the second detected request, and, after detecting (e.g., withdetecting unit 1218) the second request to configure the second userinterface, enable transmitting (e.g., with transmitting enabling unit1220) instructions to the second peripheral display unit to display thesecond user interface in accordance with the detected request.

In some embodiments, displaying on the display of the device the secondupdated representation of the second user interface occurs while a dataconnection corresponding to the relationship between the device and thesecond peripheral display unit is not active, and transmitting secondinstructions to the second peripheral display unit occurs when the dataconnection corresponding to the relationship between the device and thesecond peripheral display unit is established (e.g., with establishingunit 1222).

In some embodiments, processing unit 1214 is further configured to store(e.g., with storing unit 1224) the second instructions in a memory ofthe device at least until the data connection corresponding to therelationship between the device and the second peripheral display unitis established (e.g., with establishing unit 1222).

In some embodiments, the first instruction and the second instructionare simultaneously stored (e.g., with storing unit 1224)) at the device.

In some embodiments, the second user interface comprises one or moreuser interface objects different from the first user interface.

In some embodiments, the second user interface comprises an arrangementof user interface objects different from the first user interface.

In some embodiments, the second peripheral display unit has one or moreof a screen size different from the first peripheral display unit, ascreen orientation different from the first peripheral display unit, anda screen resolution different from the first peripheral display unit.

In some embodiments, the configuration user interface comprises a secondset of representations of one or more user interface objects availableto be added to the second user interface, the second instructionscomprise instructions for adding to the user interface a user interfaceobject corresponding to one of the one or more representations of userinterface objects from the second set, the second request to configurethe second user interface comprises second user input corresponding to alocation of the one of the one or more representations of user interfaceobjects from the second set, and the first set and the second setcomprise representations of different user interface objects.

The units of FIGS. 10-12 can optionally be used to implement the varioustechniques and methods described above. The units of devices 1000, 1100,and 1200 are, optionally, implemented by hardware, software, or acombination of hardware and software to carry out the principles of thevarious described examples. It is understood by persons of skill in theart that the functional blocks described in FIGS. 10-12 are, optionally,combined or separated into sub-blocks to implement the principles of thevarious described examples. For example, units 1002-1010, 1102-1110, and1202-1210 can have associated “controller” units that are operativelycoupled with the respective unit and processing unit to enableoperation. These controller units are not separately illustrated inFIGS. 10-12 but are understood to be within the grasp of one of ordinaryskill in the art who is designing a device having units such as those indevices 1000, 1100, or 1200. The description herein thus optionallysupports combination, separation, and/or further definition of thefunctional blocks described herein.

Although the disclosure and examples have been fully described withreference to the accompanying figures, it is to be noted that variouschanges and modifications will become apparent to those skilled in theart. Such changes and modifications are to be understood as beingincluded within the scope of the disclosure and examples as defined bythe appended claims.

The operations in the information processing methods described aboveare, optionally, implemented by running one or more functional modulesin information processing apparatus such as general purpose processors(e.g., as described above with respect to FIGS. 1A and 3) orapplication-specific chips.

The operations described above with reference to FIGS. 7A-7H, 8A-8D, and9A-9D are, optionally, implemented by components depicted in FIGS. 1A-1Bor FIG. 10, 11, or 12. For example, receiving operation 702,establishing operation 704, detecting operation 708, establishingoperation 710, receiving operation 716, displaying operation 718,displaying operation 720, displaying operation 722, displaying operation724, detecting operation 734, displaying operation 736, detectingoperation 744, displaying operation 746, transferring operation 768,detecting operation 780, activating operation 782, monitoring operation802, providing operation 808, providing operation 810, determiningoperation 824, providing operation 830, detecting operation 832,deactivating operation 834, deactivating operation 836, establishingoperation 902, displaying operation 904, detecting operation 906,displaying operation 908, transmitting operation 912, storing operation916, detecting operation 926, displaying operation 928, transmittingoperation 934, and/or storing operation 938 are, optionally, implementedby event sorter 170, event recognizer 180, and event handler 190. Eventmonitor 171 in event sorter 170 detects a contact on touch-sensitivedisplay 112, and event dispatcher module 174 delivers the eventinformation to application 136-1. A respective event recognizer 180 ofapplication 136-1 compares the event information to respective eventdefinitions 186, and determines whether a first contact at a firstlocation on the touch-sensitive surface (or whether rotation of thedevice) corresponds to a predefined event or sub-event, such asselection of an object on a user interface, or rotation of the devicefrom one orientation to another. When a respective predefined event orsub-event is detected, event recognizer 180 activates an event handler190 associated with the detection of the event or sub-event. Eventhandler 190 optionally uses or calls data updater 176 or object updater177 to update the application internal state 192. In some embodiments,event handler 190 accesses a respective GUI updater 178 to update whatis displayed by the application. Similarly, it would be clear to aperson having ordinary skill in the art how other processes can beimplemented based on the components depicted in FIGS. 1A-1B.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, to therebyenable others skilled in the art to best use the invention and variousdescribed embodiments with various modifications as are suited to theparticular use contemplated.

What is claimed is:
 1. An electronic device, comprising: one or moreprocessors; a memory; and one or more programs, wherein the one or moreprograms are stored in the memory and configured to be executed by theone or more processors, the one or more program including instructionsfor: activating a first mode of operation of the electronic devicewherein the first mode of operation comprises a wireless connectionbetween the electronic device and a peripheral display unit; monitoringbattery usage of the electronic device, wherein monitoring battery usageof the electronic device includes monitoring a charge level of one ormore batteries of the electronic device and monitoring battery usagepatterns of the one or more batteries of the electronic device sinceactivating the first mode of operation; and while monitoring batteryusage of the electronic device: in accordance with a determination thata charge level of the electronic device meets charge-level notificationcriteria, providing a charge-level alert that indicates a current chargelevel of the one or more batteries; and in accordance with adetermination that battery usage of the electronic device sinceactivating the first mode of operation meets battery-usage notificationcriteria different from the charge-level notification criteria,providing a battery-usage alert that indicates a current battery usagepattern since activating the first mode of operation.
 2. The electronicdevice of claim 1, wherein the battery usage patterns being monitoredinclude one or more of: battery amount used since activating the firstmode of operation, rate of battery usage since activating the first modeof operation, predicted time of the one or more batteries reaching apredetermined threshold level, and predicted battery level by the timeof a future activation of the first mode of operation.
 3. The electronicdevice of claim 1, wherein one or more of the charge-level alert and thebattery-usage alert are provided through the peripheral display unit. 4.The electronic device of claim 1, wherein one or more of thecharge-level alert and the battery-usage alert are provided through theperipheral display unit only when the device is not being charged. 5.The electronic device of claim 1, wherein the battery-usage notificationcriteria comprise a criterion that is met when a charge level of the oneor more batteries of the device will be below a predefined charge levelat a predicted end time of usage of the first mode of operation.
 6. Theelectronic device of claim 1, wherein the battery-usage notificationcriteria comprise a criterion that is met when a charge level of the oneor more batteries of the device will be below a predefined charge levelat time of a predefined future event.
 7. The electronic device of claim1, the one or more programs further including instructions for:determining a destination for the device, wherein the battery-usagenotification criteria comprise a criterion that is met when a chargelevel of the one or more batteries of the device will be below apredefined charge level at a predicted time at which the device willarrive at the destination.
 8. The electronic device of claim 7, whereinthe destination is a destination explicitly identified by a user.
 9. Theelectronic device of claim 7, wherein the destination is a destinationpredicted based on user behavior.
 10. The electronic device of claim 1,the one or more programs further including instructions for: inaccordance with providing one or more of the charge-level alert and thebattery-usage alert, providing an option to deactivate the first mode ofoperation; detecting selection of the option; and in accordance withdetecting selection of the option, deactivating the first mode ofoperation.
 11. The electronic device of claim 1, the one or moreprograms further including instructions for: in accordance with adetermination that a charge level of the device meets charge-levelnotification criteria, automatically deactivating the first mode ofoperation.
 12. A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructionswhich, when executed by one or more processors of an electronic device,cause the electronic device to: activating a first mode of operation ofthe electronic device wherein the first mode of operation comprises awireless connection between the electronic device and a peripheraldisplay unit; monitoring battery usage of the electronic device, whereinmonitoring battery usage of the electronic device includes monitoring acharge level of one or more batteries of the electronic device andmonitoring battery usage patterns of the one or more batteries of theelectronic device since activating the first mode of operation; andwhile monitoring battery usage of the electronic device: in accordancewith a determination that a charge level of the electronic device meetscharge-level notification criteria, providing a charge-level alert thatindicates a current charge level of the one or more batteries; and inaccordance with a determination that battery usage of the electronicdevice since activating the first mode of operation meets battery-usagenotification criteria different from the charge-level notificationcriteria, providing a battery-usage alert that indicates a currentbattery usage pattern since activating the first mode of operation. 13.The non-transitory computer readable storage medium of claim 12, whereinthe battery usage patterns being monitored include one or more of:battery amount used since activating the first mode of operation, rateof battery usage since activating the first mode of operation, predictedtime of the one or more batteries reaching a predetermined thresholdlevel, and predicted battery level by the time of a future activation ofthe first mode of operation.
 14. The non-transitory computer readablestorage medium of claim 12, wherein one or more of the charge-levelalert and the battery-usage alert are provided through the peripheraldisplay unit.
 15. The non-transitory computer readable storage medium ofclaim 12, wherein one or more of the charge-level alert and thebattery-usage alert are provided through the peripheral display unitonly when the device is not being charged.
 16. The non-transitorycomputer readable storage medium of claim 12, wherein the battery-usagenotification criteria comprise a criterion that is met when a chargelevel of the one or more batteries of the device will be below apredefined charge level at a predicted end time of usage of the firstmode of operation.
 17. The non-transitory computer readable storagemedium of claim 12, wherein the battery-usage notification criteriacomprise a criterion that is met when a charge level of the one or morebatteries of the device will be below a predefined charge level at timeof a predefined future event.
 18. The non-transitory computer readablestorage medium of claim 12, the one or more programs further includinginstructions for: determining a destination for the device, wherein thebattery-usage notification criteria comprise a criterion that is metwhen a charge level of the one or more batteries of the device will bebelow a predefined charge level at a predicted time at which the devicewill arrive at the destination.
 19. The non-transitory computer readablestorage medium of claim 18, wherein the destination is a destinationexplicitly identified by a user.
 20. The non-transitory computerreadable storage medium of claim 18, wherein the destination is adestination predicted based on user behavior.
 21. The non-transitorycomputer readable storage medium of claim 12, the one or more programsfurther including instructions for: in accordance with providing one ormore of the charge-level alert and the battery-usage alert, providing anoption to deactivate the first mode of operation; detecting selection ofthe option; and in accordance with detecting selection of the option,deactivating the first mode of operation.
 22. The non-transitorycomputer readable storage medium of claim 12, the one or more programsfurther including instructions for: in accordance with a determinationthat a charge level of the device meets charge-level notificationcriteria, automatically deactivating the first mode of operation.
 23. Amethod comprising: at an electronic device with one or more processorsand memory: activating a first mode of operation of the electronicdevice wherein the first mode of operation comprises a wirelessconnection between the electronic device and a peripheral display unit;monitoring battery usage of the electronic device, wherein monitoringbattery usage of the electronic device includes monitoring a chargelevel of one or more batteries of the electronic device and monitoringbattery usage patterns of the one or more batteries of the electronicdevice since activating the first mode of operation; and whilemonitoring battery usage of the electronic device: in accordance with adetermination that a charge level of the electronic device meetscharge-level notification criteria, providing a charge-level alert thatindicates a current charge level of the one or more batteries; and inaccordance with a determination that battery usage of the electronicdevice since activating the first mode of operation meets battery-usagenotification criteria different from the charge-level notificationcriteria, providing a battery-usage alert that indicates a currentbattery usage pattern since activating the first mode of operation. 24.The method of claim 23, wherein the battery usage patterns beingmonitored include one or more of: battery amount used since activatingthe first mode of operation, rate of battery usage since activating thefirst mode of operation, predicted time of the one or more batteriesreaching a predetermined threshold level, and predicted battery level bythe time of a future activation of the first mode of operation.
 25. Themethod of claim 23, wherein one or more of the charge-level alert andthe battery-usage alert are provided through the peripheral displayunit.
 26. The method of claim 23, wherein one or more of thecharge-level alert and the battery-usage alert are provided through theperipheral display unit only when the device is not being charged. 27.The method of claim 23, wherein the battery-usage notification criteriacomprise a criterion that is met when a charge level of the one or morebatteries of the device will be below a predefined charge level at apredicted end time of usage of the first mode of operation.
 28. Themethod of claim 23, wherein the battery-usage notification criteriacomprise a criterion that is met when a charge level of the one or morebatteries of the device will be below a predefined charge level at timeof a predefined future event.
 29. The method of claim 23, furthercomprising: determining a destination for the device, wherein thebattery-usage notification criteria comprise a criterion that is metwhen a charge level of the one or more batteries of the device will bebelow a predefined charge level at a predicted time at which the devicewill arrive at the destination.
 30. The method of claim 29, wherein thedestination is a destination explicitly identified by a user.
 31. Themethod of claim 29, wherein the destination is a destination predictedbased on user behavior.
 32. The method of claim 23, further comprising:in accordance with providing one or more of the charge-level alert andthe battery-usage alert, providing an option to deactivate the firstmode of operation; detecting selection of the option; and in accordancewith detecting selection of the option, deactivating the first mode ofoperation.
 33. The method of claim 23, further comprising: in accordancewith a determination that a charge level of the device meetscharge-level notification criteria, automatically deactivating the firstmode of operation.